2024-09-14T06:53:54.892 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T06:53:54.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T06:53:54.998 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414 branch: main description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900414' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: main ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 5839 sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 sleep_before_teardown: 0 subset: 99/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBHbPSdA5gs4Neklbw15b6Jz0kJPGBLCUU3u8DFEsXeoizVurScii8GxNVVsJol9YXZfl9DN+F69uPSbobVGnTo= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpbnw5uEgZQzlNMHRPssbUxIFZEHxLuZQarXi/6EcTsNNB7SW7YIBOiotcMpNvecbxNk9piwnGEmL7LLfapEwA= smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIEoZkAZgAE4Znx+kpj+O1M3bwG6hqbxJzoPMfn63ERTK3xKIN8oJYCSpp41Ytqsv+Hc8arLujyCZTKBHRIB2o= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_20:16:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T06:53:54.998 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa; will attempt to use it 2024-09-14T06:53:54.999 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa/tasks 2024-09-14T06:53:54.999 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T06:53:55.001 INFO:teuthology.task.internal:Checking packages... 2024-09-14T06:53:55.028 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '7f24e589328ae9e639c0009d93340e5b235d0a47' 2024-09-14T06:53:55.028 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T06:53:55.028 INFO:teuthology.packaging:ref: None 2024-09-14T06:53:55.028 INFO:teuthology.packaging:tag: None 2024-09-14T06:53:55.028 INFO:teuthology.packaging:branch: main 2024-09-14T06:53:55.028 INFO:teuthology.packaging:sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:53:55.029 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=main 2024-09-14T06:53:55.259 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4907-ga2782315-1jammy 2024-09-14T06:53:55.261 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T06:53:55.267 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T06:53:55.268 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T06:53:55.284 INFO:teuthology.task.internal:Saving configuration 2024-09-14T06:53:55.297 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T06:53:55.304 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T06:53:55.329 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 06:48:23.626840', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFoUd8S0NgkvJz3sGFjrOwoc06x6Iq48VVyBxSfivxrG'} 2024-09-14T06:53:55.349 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 06:48:23.623904', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKy6BpkPkXaMzCLlzAiAXDQEa/W0rSPzpILyz9QQmZkJ'} 2024-09-14T06:53:55.373 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 06:48:23.625544', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII/V0ZY51TfgGtWIkyki3x2NX3V3fj7IZYIYRmjav6qm'} 2024-09-14T06:53:55.373 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T06:53:55.379 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-14T06:53:55.379 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-14T06:53:55.379 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['client.0'] 2024-09-14T06:53:55.380 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T06:53:55.472 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0838463010>, signals=[15]) 2024-09-14T06:53:55.472 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T06:53:55.478 INFO:teuthology.task.internal:Opening connections... 2024-09-14T06:53:55.478 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T06:53:55.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:53:55.559 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T06:53:55.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:53:55.638 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-09-14T06:53:55.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:53:55.717 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T06:53:55.723 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-09-14T06:53:55.729 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-09-14T06:53:55.729 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Ubuntu" 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="22.04" 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_CODENAME=jammy 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:ID=ubuntu 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE=debian 2024-09-14T06:53:55.778 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T06:53:55.779 INFO:teuthology.orchestra.run.smithi005.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T06:53:55.779 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T06:53:55.779 INFO:teuthology.orchestra.run.smithi005.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T06:53:55.779 INFO:teuthology.orchestra.run.smithi005.stdout:UBUNTU_CODENAME=jammy 2024-09-14T06:53:55.779 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-09-14T06:53:55.807 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-14T06:53:55.814 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-14T06:53:55.814 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Ubuntu" 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="22.04" 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_CODENAME=jammy 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:ID=ubuntu 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE=debian 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T06:53:55.861 INFO:teuthology.orchestra.run.smithi110.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T06:53:55.862 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T06:53:55.862 INFO:teuthology.orchestra.run.smithi110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T06:53:55.862 INFO:teuthology.orchestra.run.smithi110.stdout:UBUNTU_CODENAME=jammy 2024-09-14T06:53:55.862 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-14T06:53:55.891 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-09-14T06:53:55.897 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-09-14T06:53:55.897 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-09-14T06:53:55.944 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-14T06:53:55.944 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Ubuntu" 2024-09-14T06:53:55.944 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="22.04" 2024-09-14T06:53:55.944 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_CODENAME=jammy 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:ID=ubuntu 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE=debian 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-14T06:53:55.945 INFO:teuthology.orchestra.run.smithi145.stdout:UBUNTU_CODENAME=jammy 2024-09-14T06:53:55.945 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-09-14T06:53:55.966 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T06:53:55.974 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T06:53:55.981 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T06:53:55.981 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:53:55.984 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:53:55.986 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:53:55.991 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T06:53:55.997 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T06:53:55.997 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:53:56.035 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:53:56.037 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:53:56.050 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T06:53:56.125 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T06:53:56.139 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T06:53:56.139 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T06:53:56.139 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:53:56.140 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:53:56.141 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:53:56.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:53:56.146 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-14T06:53:56.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:53:56.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:53:56.148 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-14T06:53:56.148 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-14T06:53:56.193 INFO:teuthology.orchestra.run.smithi110.stdout:5.15.0-56-generic 2024-09-14T06:53:56.193 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.15.0-56-generic 2024-09-14T06:53:56.193 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get clean 2024-09-14T06:53:56.198 INFO:teuthology.orchestra.run.smithi005.stdout:5.15.0-56-generic 2024-09-14T06:53:56.198 INFO:teuthology.orchestra.run.smithi145.stdout:5.15.0-56-generic 2024-09-14T06:53:56.198 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.15.0-56-generic 2024-09-14T06:53:56.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get clean 2024-09-14T06:53:56.199 INFO:teuthology.task.kernel:Running kernel on smithi145: 5.15.0-56-generic 2024-09-14T06:53:56.199 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-get clean 2024-09-14T06:53:56.306 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get update 2024-09-14T06:53:56.318 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get update 2024-09-14T06:53:56.330 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-get update 2024-09-14T06:53:56.442 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:53:56.464 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T06:53:56.526 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T06:53:56.580 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T06:53:56.593 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:53:56.595 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:53:56.595 INFO:teuthology.orchestra.run.smithi145.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T06:53:56.601 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-14T06:53:56.654 INFO:teuthology.orchestra.run.smithi005.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T06:53:56.656 INFO:teuthology.orchestra.run.smithi145.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-14T06:53:56.673 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T06:53:56.741 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T06:53:56.776 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T06:53:56.779 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T06:53:56.779 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T06:53:56.805 INFO:teuthology.orchestra.run.smithi110.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T06:53:56.805 INFO:teuthology.orchestra.run.smithi110.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T06:53:56.809 INFO:teuthology.orchestra.run.smithi110.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T06:53:56.809 INFO:teuthology.orchestra.run.smithi110.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T06:53:56.814 INFO:teuthology.orchestra.run.smithi110.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T06:53:56.821 INFO:teuthology.orchestra.run.smithi110.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T06:53:56.822 INFO:teuthology.orchestra.run.smithi110.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T06:53:56.823 INFO:teuthology.orchestra.run.smithi110.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T06:53:56.823 INFO:teuthology.orchestra.run.smithi110.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T06:53:56.834 INFO:teuthology.orchestra.run.smithi110.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T06:53:56.856 INFO:teuthology.orchestra.run.smithi110.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T06:53:56.857 INFO:teuthology.orchestra.run.smithi110.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T06:53:56.857 INFO:teuthology.orchestra.run.smithi110.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T06:53:56.857 INFO:teuthology.orchestra.run.smithi110.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T06:53:56.858 INFO:teuthology.orchestra.run.smithi110.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T06:53:56.858 INFO:teuthology.orchestra.run.smithi110.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T06:53:56.859 INFO:teuthology.orchestra.run.smithi110.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T06:53:56.859 INFO:teuthology.orchestra.run.smithi110.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T06:53:56.859 INFO:teuthology.orchestra.run.smithi110.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T06:53:57.010 INFO:teuthology.orchestra.run.smithi110.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T06:53:57.010 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T06:53:57.011 INFO:teuthology.orchestra.run.smithi145.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-14T06:53:57.031 INFO:teuthology.orchestra.run.smithi145.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T06:53:57.040 INFO:teuthology.orchestra.run.smithi005.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T06:53:57.147 INFO:teuthology.orchestra.run.smithi005.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T06:53:57.150 INFO:teuthology.orchestra.run.smithi145.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T06:53:57.324 INFO:teuthology.orchestra.run.smithi145.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-14T06:53:57.344 INFO:teuthology.orchestra.run.smithi145.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-14T06:53:57.431 INFO:teuthology.orchestra.run.smithi110.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T06:53:57.447 INFO:teuthology.orchestra.run.smithi110.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T06:53:57.456 INFO:teuthology.orchestra.run.smithi110.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T06:53:57.456 INFO:teuthology.orchestra.run.smithi110.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T06:53:57.465 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-14T06:53:57.471 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-14T06:53:57.488 INFO:teuthology.orchestra.run.smithi005.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T06:53:57.496 INFO:teuthology.orchestra.run.smithi005.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T06:53:57.497 INFO:teuthology.orchestra.run.smithi145.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-14T06:53:57.497 INFO:teuthology.orchestra.run.smithi005.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T06:53:57.500 INFO:teuthology.orchestra.run.smithi005.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T06:53:57.503 INFO:teuthology.orchestra.run.smithi145.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-14T06:53:57.509 INFO:teuthology.orchestra.run.smithi145.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-14T06:53:57.510 INFO:teuthology.orchestra.run.smithi145.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T06:53:57.512 INFO:teuthology.orchestra.run.smithi145.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-14T06:53:57.514 INFO:teuthology.orchestra.run.smithi005.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T06:53:57.516 INFO:teuthology.orchestra.run.smithi005.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T06:53:57.517 INFO:teuthology.orchestra.run.smithi145.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-14T06:53:57.518 INFO:teuthology.orchestra.run.smithi005.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T06:53:57.518 INFO:teuthology.orchestra.run.smithi145.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-14T06:53:57.519 INFO:teuthology.orchestra.run.smithi145.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-14T06:53:57.552 INFO:teuthology.orchestra.run.smithi110.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T06:53:57.554 INFO:teuthology.orchestra.run.smithi110.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T06:53:57.564 INFO:teuthology.orchestra.run.smithi110.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T06:53:57.565 INFO:teuthology.orchestra.run.smithi110.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T06:53:57.585 INFO:teuthology.orchestra.run.smithi005.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T06:53:57.588 INFO:teuthology.orchestra.run.smithi110.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T06:53:57.593 INFO:teuthology.orchestra.run.smithi005.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-14T06:53:57.594 INFO:teuthology.orchestra.run.smithi005.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T06:53:57.601 INFO:teuthology.orchestra.run.smithi005.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T06:53:57.601 INFO:teuthology.orchestra.run.smithi005.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T06:53:57.610 INFO:teuthology.orchestra.run.smithi110.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T06:53:57.613 INFO:teuthology.orchestra.run.smithi110.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T06:53:57.616 INFO:teuthology.orchestra.run.smithi110.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T06:53:57.617 INFO:teuthology.orchestra.run.smithi110.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T06:53:57.622 INFO:teuthology.orchestra.run.smithi145.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-14T06:53:57.622 INFO:teuthology.orchestra.run.smithi145.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-14T06:53:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T06:53:57.627 INFO:teuthology.orchestra.run.smithi145.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T06:53:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-14T06:53:57.630 INFO:teuthology.orchestra.run.smithi145.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-14T06:53:57.636 INFO:teuthology.orchestra.run.smithi145.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T06:53:57.649 INFO:teuthology.orchestra.run.smithi005.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T06:53:57.649 INFO:teuthology.orchestra.run.smithi005.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-14T06:53:57.649 INFO:teuthology.orchestra.run.smithi005.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-14T06:53:57.650 INFO:teuthology.orchestra.run.smithi145.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-14T06:53:57.653 INFO:teuthology.orchestra.run.smithi145.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T06:53:57.659 INFO:teuthology.orchestra.run.smithi145.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T06:53:57.660 INFO:teuthology.orchestra.run.smithi145.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T06:53:57.661 INFO:teuthology.orchestra.run.smithi145.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T06:53:57.661 INFO:teuthology.orchestra.run.smithi145.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T06:53:57.661 INFO:teuthology.orchestra.run.smithi005.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T06:53:57.662 INFO:teuthology.orchestra.run.smithi145.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-14T06:53:57.664 INFO:teuthology.orchestra.run.smithi005.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-14T06:53:57.665 INFO:teuthology.orchestra.run.smithi145.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T06:53:57.668 INFO:teuthology.orchestra.run.smithi145.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T06:53:57.668 INFO:teuthology.orchestra.run.smithi005.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-14T06:53:57.668 INFO:teuthology.orchestra.run.smithi005.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-14T06:53:57.669 INFO:teuthology.orchestra.run.smithi145.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T06:53:57.669 INFO:teuthology.orchestra.run.smithi145.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T06:53:57.669 INFO:teuthology.orchestra.run.smithi005.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-14T06:53:57.669 INFO:teuthology.orchestra.run.smithi005.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-14T06:53:57.675 INFO:teuthology.orchestra.run.smithi005.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-14T06:53:57.678 INFO:teuthology.orchestra.run.smithi005.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-14T06:53:57.679 INFO:teuthology.orchestra.run.smithi005.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-14T06:53:57.679 INFO:teuthology.orchestra.run.smithi005.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-14T06:53:57.692 INFO:teuthology.orchestra.run.smithi110.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T06:53:57.693 INFO:teuthology.orchestra.run.smithi110.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T06:53:57.714 INFO:teuthology.orchestra.run.smithi005.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T06:53:57.714 INFO:teuthology.orchestra.run.smithi005.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T06:53:57.715 INFO:teuthology.orchestra.run.smithi005.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T06:53:57.736 INFO:teuthology.orchestra.run.smithi005.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-14T06:53:57.736 INFO:teuthology.orchestra.run.smithi005.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T06:53:57.742 INFO:teuthology.orchestra.run.smithi145.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-14T06:53:57.748 INFO:teuthology.orchestra.run.smithi145.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-14T06:53:57.749 INFO:teuthology.orchestra.run.smithi145.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-14T06:53:57.750 INFO:teuthology.orchestra.run.smithi145.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-14T06:53:57.751 INFO:teuthology.orchestra.run.smithi005.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T06:53:57.752 INFO:teuthology.orchestra.run.smithi145.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-14T06:53:57.752 INFO:teuthology.orchestra.run.smithi145.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T06:53:57.752 INFO:teuthology.orchestra.run.smithi145.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T06:53:57.752 INFO:teuthology.orchestra.run.smithi005.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-14T06:53:57.752 INFO:teuthology.orchestra.run.smithi005.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T06:53:57.753 INFO:teuthology.orchestra.run.smithi145.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-14T06:53:57.753 INFO:teuthology.orchestra.run.smithi005.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-14T06:53:57.753 INFO:teuthology.orchestra.run.smithi005.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T06:53:57.795 INFO:teuthology.orchestra.run.smithi005.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T06:53:57.831 INFO:teuthology.orchestra.run.smithi145.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-14T06:53:57.831 INFO:teuthology.orchestra.run.smithi145.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-14T06:54:03.118 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 16.0 MB in 2s (7,412 kB/s) 2024-09-14T06:54:03.700 INFO:teuthology.orchestra.run.smithi145.stdout:Fetched 16.0 MB in 3s (5,821 kB/s) 2024-09-14T06:54:04.129 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 16.0 MB in 3s (5,489 kB/s) 2024-09-14T06:54:04.547 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-14T06:54:04.564 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:54:04.674 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-14T06:54:04.893 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-14T06:54:04.894 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-14T06:54:05.087 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:54:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:54:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:54:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:54:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T06:54:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: python2.7-minimal 2024-09-14T06:54:05.089 INFO:teuthology.orchestra.run.smithi110.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:54:05.090 INFO:teuthology.orchestra.run.smithi110.stdout:The following additional packages will be installed: 2024-09-14T06:54:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:54:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T06:54:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T06:54:05.092 INFO:teuthology.orchestra.run.smithi110.stdout:Suggested packages: 2024-09-14T06:54:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T06:54:05.092 INFO:teuthology.orchestra.run.smithi110.stdout:Recommended packages: 2024-09-14T06:54:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: thermald 2024-09-14T06:54:05.141 INFO:teuthology.orchestra.run.smithi110.stdout:The following NEW packages will be installed: 2024-09-14T06:54:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:54:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T06:54:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T06:54:05.143 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages will be upgraded: 2024-09-14T06:54:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T06:54:05.153 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-14T06:54:05.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:54:05.225 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-14T06:54:05.322 INFO:teuthology.orchestra.run.smithi110.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:54:05.322 INFO:teuthology.orchestra.run.smithi110.stdout:Need to get 113 MB of archives. 2024-09-14T06:54:05.323 INFO:teuthology.orchestra.run.smithi110.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T06:54:05.323 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T06:54:05.419 INFO:teuthology.orchestra.run.smithi145.stdout:Building dependency tree... 2024-09-14T06:54:05.420 INFO:teuthology.orchestra.run.smithi145.stdout:Reading state information... 2024-09-14T06:54:05.593 INFO:teuthology.orchestra.run.smithi145.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:54:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:54:05.593 INFO:teuthology.orchestra.run.smithi145.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:54:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:54:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T06:54:05.594 INFO:teuthology.orchestra.run.smithi145.stdout: python2.7-minimal 2024-09-14T06:54:05.594 INFO:teuthology.orchestra.run.smithi145.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:54:05.595 INFO:teuthology.orchestra.run.smithi145.stdout:The following additional packages will be installed: 2024-09-14T06:54:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:54:05.595 INFO:teuthology.orchestra.run.smithi145.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T06:54:05.596 INFO:teuthology.orchestra.run.smithi145.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T06:54:05.597 INFO:teuthology.orchestra.run.smithi145.stdout:Suggested packages: 2024-09-14T06:54:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T06:54:05.597 INFO:teuthology.orchestra.run.smithi145.stdout:Recommended packages: 2024-09-14T06:54:05.597 INFO:teuthology.orchestra.run.smithi145.stdout: thermald 2024-09-14T06:54:05.642 INFO:teuthology.orchestra.run.smithi145.stdout:The following NEW packages will be installed: 2024-09-14T06:54:05.642 INFO:teuthology.orchestra.run.smithi145.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:54:05.642 INFO:teuthology.orchestra.run.smithi145.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T06:54:05.643 INFO:teuthology.orchestra.run.smithi145.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T06:54:05.643 INFO:teuthology.orchestra.run.smithi145.stdout:The following packages will be upgraded: 2024-09-14T06:54:05.643 INFO:teuthology.orchestra.run.smithi145.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T06:54:05.657 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-09-14T06:54:05.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:54:05.695 INFO:teuthology.orchestra.run.smithi145.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:54:05.695 INFO:teuthology.orchestra.run.smithi145.stdout:Need to get 113 MB of archives. 2024-09-14T06:54:05.695 INFO:teuthology.orchestra.run.smithi145.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T06:54:05.695 INFO:teuthology.orchestra.run.smithi145.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T06:54:05.731 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-09-14T06:54:05.921 INFO:teuthology.orchestra.run.smithi005.stdout:Building dependency tree... 2024-09-14T06:54:05.922 INFO:teuthology.orchestra.run.smithi005.stdout:Reading state information... 2024-09-14T06:54:06.095 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:54:06.095 INFO:teuthology.orchestra.run.smithi005.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:54:06.096 INFO:teuthology.orchestra.run.smithi005.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:54:06.096 INFO:teuthology.orchestra.run.smithi005.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:54:06.096 INFO:teuthology.orchestra.run.smithi005.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-14T06:54:06.096 INFO:teuthology.orchestra.run.smithi005.stdout: python2.7-minimal 2024-09-14T06:54:06.096 INFO:teuthology.orchestra.run.smithi005.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:54:06.097 INFO:teuthology.orchestra.run.smithi005.stdout:The following additional packages will be installed: 2024-09-14T06:54:06.097 INFO:teuthology.orchestra.run.smithi005.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:54:06.097 INFO:teuthology.orchestra.run.smithi005.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-14T06:54:06.097 INFO:teuthology.orchestra.run.smithi005.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-14T06:54:06.098 INFO:teuthology.orchestra.run.smithi145.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T06:54:06.099 INFO:teuthology.orchestra.run.smithi005.stdout:Suggested packages: 2024-09-14T06:54:06.099 INFO:teuthology.orchestra.run.smithi005.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-14T06:54:06.099 INFO:teuthology.orchestra.run.smithi005.stdout:Recommended packages: 2024-09-14T06:54:06.099 INFO:teuthology.orchestra.run.smithi005.stdout: thermald 2024-09-14T06:54:06.144 INFO:teuthology.orchestra.run.smithi005.stdout:The following NEW packages will be installed: 2024-09-14T06:54:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-14T06:54:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-14T06:54:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-14T06:54:06.145 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages will be upgraded: 2024-09-14T06:54:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-14T06:54:06.190 INFO:teuthology.orchestra.run.smithi145.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T06:54:06.197 INFO:teuthology.orchestra.run.smithi005.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:54:06.197 INFO:teuthology.orchestra.run.smithi005.stdout:Need to get 113 MB of archives. 2024-09-14T06:54:06.197 INFO:teuthology.orchestra.run.smithi005.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-14T06:54:06.197 INFO:teuthology.orchestra.run.smithi005.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-14T06:54:06.652 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T06:54:06.738 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-14T06:54:06.777 INFO:teuthology.orchestra.run.smithi005.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T06:54:07.034 INFO:teuthology.orchestra.run.smithi145.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T06:54:07.066 INFO:teuthology.orchestra.run.smithi145.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T06:54:07.069 INFO:teuthology.orchestra.run.smithi145.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T06:54:07.087 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-14T06:54:07.219 INFO:teuthology.orchestra.run.smithi145.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T06:54:07.242 INFO:teuthology.orchestra.run.smithi145.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T06:54:07.620 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T06:54:07.620 INFO:teuthology.orchestra.run.smithi005.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T06:54:07.620 INFO:teuthology.orchestra.run.smithi005.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T06:54:07.625 INFO:teuthology.orchestra.run.smithi145.stdout:Fetched 113 MB in 2s (71.2 MB/s) 2024-09-14T06:54:07.752 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T06:54:07.774 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T06:54:07.809 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T06:54:08.265 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 113 MB in 2s (67.0 MB/s) 2024-09-14T06:54:08.446 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T06:54:09.352 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-14T06:54:09.353 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-14T06:54:09.353 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-14T06:54:09.705 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-14T06:54:09.788 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-14T06:54:09.862 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T06:54:09.866 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:09.957 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:10.208 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 113 MB in 5s (24.4 MB/s) 2024-09-14T06:54:10.383 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-14T06:54:10.444 INFO:teuthology.orchestra.run.smithi005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T06:54:10.448 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:10.519 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:12.375 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-14T06:54:12.379 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:12.473 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:12.528 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T06:54:12.551 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:12.613 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T06:54:12.630 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:12.649 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:12.732 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:13.123 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T06:54:13.144 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:13.173 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:13.203 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T06:54:13.224 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:13.253 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:14.609 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-14T06:54:14.630 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:14.703 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:15.187 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-14T06:54:15.208 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:15.245 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:19.748 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:19.858 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:19.925 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:20.051 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:20.234 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:20.362 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:20.367 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:20.471 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:20.681 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T06:54:20.704 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T06:54:20.740 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:54:20.799 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T06:54:20.810 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T06:54:20.849 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:54:22.414 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:22.559 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:22.824 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:22.929 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:23.241 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-14T06:54:23.263 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-14T06:54:23.307 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:54:26.103 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T06:54:26.141 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:26.186 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:26.821 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T06:54:26.858 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:26.895 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:29.129 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-14T06:54:29.165 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-14T06:54:29.229 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:29.412 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:29.534 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:29.619 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:29.733 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:29.901 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:54:30.014 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:30.057 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:54:30.132 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T06:54:30.154 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:30.268 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:30.280 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T06:54:30.408 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:31.566 INFO:teuthology.orchestra.run.smithi145.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:31.567 INFO:teuthology.orchestra.run.smithi145.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:31.567 INFO:teuthology.orchestra.run.smithi145.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:31.567 INFO:teuthology.orchestra.run.smithi145.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:31.754 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:31.782 INFO:teuthology.orchestra.run.smithi005.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:31.782 INFO:teuthology.orchestra.run.smithi005.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:31.782 INFO:teuthology.orchestra.run.smithi005.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:31.783 INFO:teuthology.orchestra.run.smithi005.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:31.945 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:32.884 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-14T06:54:32.985 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-14T06:54:33.317 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-14T06:54:33.422 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:33.429 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:33.456 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:33.582 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-14T06:54:33.710 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:34.839 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T06:54:34.847 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T06:54:34.950 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T06:54:34.965 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T06:54:35.030 INFO:teuthology.orchestra.run.smithi110.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:35.030 INFO:teuthology.orchestra.run.smithi110.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:35.030 INFO:teuthology.orchestra.run.smithi110.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:35.031 INFO:teuthology.orchestra.run.smithi110.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:35.051 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:35.092 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:35.122 INFO:teuthology.orchestra.run.smithi145.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T06:54:35.123 INFO:teuthology.orchestra.run.smithi145.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:35.166 INFO:teuthology.orchestra.run.smithi005.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T06:54:35.166 INFO:teuthology.orchestra.run.smithi005.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:35.197 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:36.599 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:38.451 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-14T06:54:38.968 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-14T06:54:39.095 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-14T06:54:39.179 INFO:teuthology.orchestra.run.smithi110.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-14T06:54:39.179 INFO:teuthology.orchestra.run.smithi110.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:49.390 INFO:teuthology.orchestra.run.smithi145.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T06:54:49.390 INFO:teuthology.orchestra.run.smithi145.stdout:Sourcing file `/etc/default/grub' 2024-09-14T06:54:49.423 INFO:teuthology.orchestra.run.smithi145.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:54:49.448 INFO:teuthology.orchestra.run.smithi145.stdout:Generating grub configuration file ... 2024-09-14T06:54:49.623 INFO:teuthology.orchestra.run.smithi005.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T06:54:49.624 INFO:teuthology.orchestra.run.smithi005.stdout:Sourcing file `/etc/default/grub' 2024-09-14T06:54:49.635 INFO:teuthology.orchestra.run.smithi005.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:54:49.661 INFO:teuthology.orchestra.run.smithi005.stdout:Generating grub configuration file ... 2024-09-14T06:54:49.752 INFO:teuthology.orchestra.run.smithi145.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:49.774 INFO:teuthology.orchestra.run.smithi145.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:49.895 INFO:teuthology.orchestra.run.smithi005.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:49.909 INFO:teuthology.orchestra.run.smithi005.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:50.121 INFO:teuthology.orchestra.run.smithi145.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:50.127 INFO:teuthology.orchestra.run.smithi145.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:50.202 INFO:teuthology.orchestra.run.smithi145.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:54:50.207 INFO:teuthology.orchestra.run.smithi145.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:54:50.226 INFO:teuthology.orchestra.run.smithi005.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:50.232 INFO:teuthology.orchestra.run.smithi005.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:50.313 INFO:teuthology.orchestra.run.smithi005.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:54:50.319 INFO:teuthology.orchestra.run.smithi005.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:54:50.353 INFO:teuthology.orchestra.run.smithi145.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:54:50.353 INFO:teuthology.orchestra.run.smithi145.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:54:50.353 INFO:teuthology.orchestra.run.smithi145.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:54:50.399 INFO:teuthology.orchestra.run.smithi145.stdout:done 2024-09-14T06:54:50.469 INFO:teuthology.orchestra.run.smithi005.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:54:50.470 INFO:teuthology.orchestra.run.smithi005.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:54:50.470 INFO:teuthology.orchestra.run.smithi005.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:54:50.508 INFO:teuthology.orchestra.run.smithi005.stdout:done 2024-09-14T06:54:50.899 DEBUG:teuthology.orchestra.run.smithi145:> dpkg -s linux-image-generic 2024-09-14T06:54:50.914 INFO:teuthology.orchestra.run.smithi145.stdout:Package: linux-image-generic 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Status: install ok installed 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Priority: optional 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Section: kernel 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Installed-Size: 21 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: amd64 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Source: linux-meta 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Version: 5.15.0.121.121 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:54:50.915 INFO:teuthology.orchestra.run.smithi145.stdout:Recommends: thermald 2024-09-14T06:54:50.916 INFO:teuthology.orchestra.run.smithi145.stdout:Description: Generic Linux kernel image 2024-09-14T06:54:50.916 INFO:teuthology.orchestra.run.smithi145.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:54:50.916 INFO:teuthology.orchestra.run.smithi145.stdout: available. 2024-09-14T06:54:50.916 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T06:54:50.916 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:54:50.916 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:54:50.917 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-14T06:54:50.917 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi145.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:54:50.917 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-get clean 2024-09-14T06:54:50.979 DEBUG:teuthology.orchestra.run.smithi005:> dpkg -s linux-image-generic 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Package: linux-image-generic 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Status: install ok installed 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Priority: optional 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Section: kernel 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Installed-Size: 21 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: amd64 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Source: linux-meta 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Version: 5.15.0.121.121 2024-09-14T06:54:50.998 INFO:teuthology.orchestra.run.smithi005.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:54:50.999 INFO:teuthology.orchestra.run.smithi005.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:54:50.999 INFO:teuthology.orchestra.run.smithi005.stdout:Recommends: thermald 2024-09-14T06:54:50.999 INFO:teuthology.orchestra.run.smithi005.stdout:Description: Generic Linux kernel image 2024-09-14T06:54:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:54:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: available. 2024-09-14T06:54:50.999 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T06:54:50.999 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:54:50.999 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:54:50.999 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T06:54:50.999 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:54:50.999 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get clean 2024-09-14T06:54:51.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-get update 2024-09-14T06:54:51.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get update 2024-09-14T06:54:51.192 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T06:54:51.278 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:54:51.287 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T06:54:51.304 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T06:54:51.314 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:54:51.396 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T06:54:51.400 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T06:54:51.476 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T06:54:52.576 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-14T06:54:52.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:54:52.650 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-14T06:54:52.683 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-09-14T06:54:52.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:54:52.768 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-09-14T06:54:52.845 INFO:teuthology.orchestra.run.smithi145.stdout:Building dependency tree... 2024-09-14T06:54:52.845 INFO:teuthology.orchestra.run.smithi145.stdout:Reading state information... 2024-09-14T06:54:52.982 INFO:teuthology.orchestra.run.smithi005.stdout:Building dependency tree... 2024-09-14T06:54:52.983 INFO:teuthology.orchestra.run.smithi005.stdout:Reading state information... 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T06:54:53.011 INFO:teuthology.orchestra.run.smithi145.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:54:53.052 INFO:teuthology.orchestra.run.smithi145.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:54:53.054 DEBUG:teuthology.orchestra.run.smithi145:> dpkg -s linux-image-generic 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Package: linux-image-generic 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Status: install ok installed 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Priority: optional 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Section: kernel 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Installed-Size: 21 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: amd64 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Source: linux-meta 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Version: 5.15.0.121.121 2024-09-14T06:54:53.109 INFO:teuthology.orchestra.run.smithi145.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:54:53.110 INFO:teuthology.orchestra.run.smithi145.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:54:53.110 INFO:teuthology.orchestra.run.smithi145.stdout:Recommends: thermald 2024-09-14T06:54:53.110 INFO:teuthology.orchestra.run.smithi145.stdout:Description: Generic Linux kernel image 2024-09-14T06:54:53.110 INFO:teuthology.orchestra.run.smithi145.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:54:53.110 INFO:teuthology.orchestra.run.smithi145.stdout: available. 2024-09-14T06:54:53.110 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-09-14T06:54:53.155 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.SQ1T9fU0nd 2024-09-14T06:54:53.155 DEBUG:teuthology.orchestra.run.smithi145:> sudo cp /boot/grub/grub.cfg /tmp/tmp.SQ1T9fU0nd 2024-09-14T06:54:53.179 INFO:teuthology.orchestra.run.smithi005.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T06:54:53.179 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:54:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:54:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:54:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:54:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T06:54:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T06:54:53.180 INFO:teuthology.orchestra.run.smithi005.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:54:53.211 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.SQ1T9fU0nd 2024-09-14T06:54:53.226 INFO:teuthology.orchestra.run.smithi005.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:54:53.228 DEBUG:teuthology.orchestra.run.smithi005:> dpkg -s linux-image-generic 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Package: linux-image-generic 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Status: install ok installed 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Priority: optional 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Section: kernel 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Installed-Size: 21 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: amd64 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Source: linux-meta 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Version: 5.15.0.121.121 2024-09-14T06:54:53.241 INFO:teuthology.orchestra.run.smithi005.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:54:53.242 INFO:teuthology.orchestra.run.smithi005.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:54:53.242 INFO:teuthology.orchestra.run.smithi005.stdout:Recommends: thermald 2024-09-14T06:54:53.242 INFO:teuthology.orchestra.run.smithi005.stdout:Description: Generic Linux kernel image 2024-09-14T06:54:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:54:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: available. 2024-09-14T06:54:53.242 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-14T06:54:53.288 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.km0rRVieKr 2024-09-14T06:54:53.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub/grub.cfg /tmp/tmp.km0rRVieKr 2024-09-14T06:54:53.336 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.SQ1T9fU0nd is 10KB 2024-09-14T06:54:53.349 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.SQ1T9fU0nd 2024-09-14T06:54:53.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T06:54:53.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.km0rRVieKr 2024-09-14T06:54:53.363 INFO:teuthology.orchestra.run.smithi110.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-14T06:54:53.363 INFO:teuthology.orchestra.run.smithi110.stdout:Sourcing file `/etc/default/grub' 2024-09-14T06:54:53.376 INFO:teuthology.orchestra.run.smithi110.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:54:53.406 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T06:54:53.406 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T06:54:53.406 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T06:54:53.439 INFO:teuthology.orchestra.run.smithi110.stdout:Generating grub configuration file ... 2024-09-14T06:54:53.475 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T06:54:53.475 DEBUG:teuthology.orchestra.run.smithi145:> sudo update-grub 2024-09-14T06:54:53.477 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.km0rRVieKr is 10KB 2024-09-14T06:54:53.489 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.km0rRVieKr 2024-09-14T06:54:53.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T06:54:53.553 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T06:54:53.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T06:54:53.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T06:54:53.621 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T06:54:53.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo update-grub 2024-09-14T06:54:53.792 INFO:teuthology.orchestra.run.smithi110.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:53.811 INFO:teuthology.orchestra.run.smithi110.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:54.197 INFO:teuthology.orchestra.run.smithi110.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:54.203 INFO:teuthology.orchestra.run.smithi110.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:54.272 INFO:teuthology.orchestra.run.smithi110.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:54:54.277 INFO:teuthology.orchestra.run.smithi110.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:54:54.439 INFO:teuthology.orchestra.run.smithi110.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:54:54.440 INFO:teuthology.orchestra.run.smithi110.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:54:54.440 INFO:teuthology.orchestra.run.smithi110.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:54:54.480 INFO:teuthology.orchestra.run.smithi110.stdout:done 2024-09-14T06:54:54.704 INFO:teuthology.orchestra.run.smithi145.stderr:Sourcing file `/etc/default/grub' 2024-09-14T06:54:54.705 INFO:teuthology.orchestra.run.smithi145.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:54:54.707 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-09-14T06:54:54.830 INFO:teuthology.orchestra.run.smithi005.stderr:Sourcing file `/etc/default/grub' 2024-09-14T06:54:54.831 INFO:teuthology.orchestra.run.smithi005.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:54:54.832 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-09-14T06:54:54.903 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:54.910 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:54.997 INFO:teuthology.orchestra.run.smithi005.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:55.002 DEBUG:teuthology.orchestra.run.smithi110:> dpkg -s linux-image-generic 2024-09-14T06:54:55.003 INFO:teuthology.orchestra.run.smithi005.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:55.028 INFO:teuthology.orchestra.run.smithi110.stdout:Package: linux-image-generic 2024-09-14T06:54:55.028 INFO:teuthology.orchestra.run.smithi110.stdout:Status: install ok installed 2024-09-14T06:54:55.028 INFO:teuthology.orchestra.run.smithi110.stdout:Priority: optional 2024-09-14T06:54:55.028 INFO:teuthology.orchestra.run.smithi110.stdout:Section: kernel 2024-09-14T06:54:55.028 INFO:teuthology.orchestra.run.smithi110.stdout:Installed-Size: 21 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: amd64 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Source: linux-meta 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Version: 5.15.0.121.121 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Recommends: thermald 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout:Description: Generic Linux kernel image 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:54:55.029 INFO:teuthology.orchestra.run.smithi110.stdout: available. 2024-09-14T06:54:55.029 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-14T06:54:55.030 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:54:55.030 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:54:55.030 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T06:54:55.030 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:54:55.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get clean 2024-09-14T06:54:55.164 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:55.169 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:55.209 INFO:teuthology.orchestra.run.smithi145.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:54:55.214 INFO:teuthology.orchestra.run.smithi145.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:54:55.222 INFO:teuthology.orchestra.run.smithi005.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:55.227 INFO:teuthology.orchestra.run.smithi005.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:55.264 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get update 2024-09-14T06:54:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:54:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:54:55.344 INFO:teuthology.orchestra.run.smithi145.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:54:55.345 INFO:teuthology.orchestra.run.smithi145.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:54:55.345 INFO:teuthology.orchestra.run.smithi145.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:54:55.364 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-09-14T06:54:55.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo shutdown -r now 2024-09-14T06:54:55.404 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T06:54:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:54:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:54:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:54:55.416 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-09-14T06:54:55.416 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T06:54:55.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-09-14T06:54:55.438 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T06:54:55.456 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T06:54:56.831 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-14T06:54:56.849 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-14T06:54:56.910 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-14T06:54:57.123 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-14T06:54:57.123 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-14T06:54:57.304 INFO:teuthology.orchestra.run.smithi110.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-14T06:54:57.304 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T06:54:57.304 INFO:teuthology.orchestra.run.smithi110.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T06:54:57.304 INFO:teuthology.orchestra.run.smithi110.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T06:54:57.304 INFO:teuthology.orchestra.run.smithi110.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T06:54:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T06:54:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T06:54:57.305 INFO:teuthology.orchestra.run.smithi110.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T06:54:57.345 INFO:teuthology.orchestra.run.smithi110.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-14T06:54:57.347 DEBUG:teuthology.orchestra.run.smithi110:> dpkg -s linux-image-generic 2024-09-14T06:54:57.362 INFO:teuthology.orchestra.run.smithi110.stdout:Package: linux-image-generic 2024-09-14T06:54:57.362 INFO:teuthology.orchestra.run.smithi110.stdout:Status: install ok installed 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Priority: optional 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Section: kernel 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Installed-Size: 21 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Maintainer: Ubuntu Kernel Team 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: amd64 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Source: linux-meta 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Version: 5.15.0.121.121 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Recommends: thermald 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout:Description: Generic Linux kernel image 2024-09-14T06:54:57.363 INFO:teuthology.orchestra.run.smithi110.stdout: This package will always depend on the latest generic kernel image 2024-09-14T06:54:57.364 INFO:teuthology.orchestra.run.smithi110.stdout: available. 2024-09-14T06:54:57.364 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-14T06:54:57.408 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.eq4sInnmzi 2024-09-14T06:54:57.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub/grub.cfg /tmp/tmp.eq4sInnmzi 2024-09-14T06:54:57.464 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.eq4sInnmzi 2024-09-14T06:54:57.589 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.eq4sInnmzi is 10KB 2024-09-14T06:54:57.601 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.eq4sInnmzi 2024-09-14T06:54:57.607 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-14T06:54:57.660 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T06:54:57.660 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-14T06:54:57.660 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-14T06:54:57.728 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-14T06:54:57.728 DEBUG:teuthology.orchestra.run.smithi110:> sudo update-grub 2024-09-14T06:54:58.959 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub' 2024-09-14T06:54:58.960 INFO:teuthology.orchestra.run.smithi110.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-14T06:54:58.963 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-14T06:54:59.161 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-14T06:54:59.168 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-14T06:54:59.499 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-14T06:54:59.505 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-14T06:54:59.548 INFO:teuthology.orchestra.run.smithi110.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-14T06:54:59.553 INFO:teuthology.orchestra.run.smithi110.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-14T06:54:59.697 INFO:teuthology.orchestra.run.smithi110.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-14T06:54:59.697 INFO:teuthology.orchestra.run.smithi110.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-14T06:54:59.697 INFO:teuthology.orchestra.run.smithi110.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-14T06:54:59.708 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-14T06:54:59.710 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-09-14T06:55:25.397 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-14T06:55:25.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-14T06:55:25.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:55:25.427 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T06:55:25.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T06:55:25.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:55:29.713 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T06:55:29.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-14T06:55:29.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:55:43.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-14T06:55:43.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-14T06:55:48.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-14T06:55:52.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-14T06:55:52.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:55:52.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T06:55:52.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:55:56.001 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-14T06:55:56.001 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-14T06:55:57.030 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-14T06:55:57.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:56:00.093 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-09-14T06:56:08.009 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T06:56:08.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:56:08.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-14T06:56:08.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:56:11.069 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-14T06:56:12.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-14T06:56:12.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:56:26.084 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T06:56:26.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:56:26.429 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-09-14T06:56:41.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-14T06:56:41.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:56:41.741 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-14T06:56:42.301 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-14T06:56:42.302 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.15.0-121-generic"... 2024-09-14T06:56:42.302 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-09-14T06:56:42.353 INFO:teuthology.orchestra.run.smithi145.stdout:5.15.0-121-generic 2024-09-14T06:56:42.353 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T06:56:42.353 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:56:42.353 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T06:56:43.354 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-14T06:56:43.354 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:56:43.429 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2024-09-14T06:56:43.452 DEBUG:teuthology.parallel:result is None 2024-09-14T06:56:43.894 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-14T06:56:44.521 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-14T06:56:44.521 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-14T06:56:44.521 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-14T06:56:44.569 INFO:teuthology.orchestra.run.smithi110.stdout:5.15.0-121-generic 2024-09-14T06:56:44.569 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T06:56:44.569 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:56:44.569 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T06:56:44.609 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-14T06:56:45.570 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T06:56:45.570 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:56:45.647 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-09-14T06:56:45.671 DEBUG:teuthology.parallel:result is None 2024-09-14T06:57:02.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T06:57:02.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:57:02.921 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-14T06:57:03.611 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T06:57:03.611 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-14T06:57:03.611 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-14T06:57:03.662 INFO:teuthology.orchestra.run.smithi005.stdout:5.15.0-121-generic 2024-09-14T06:57:03.662 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-14T06:57:03.662 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:57:03.662 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-14T06:57:04.663 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T06:57:04.664 DEBUG:teuthology.orchestra.run.smithi005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:57:04.753 INFO:teuthology.orchestra.run.smithi005.stdout:ttyS1 2024-09-14T06:57:04.779 DEBUG:teuthology.parallel:result is None 2024-09-14T06:57:04.779 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T06:57:04.787 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T06:57:04.787 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:57:04.790 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:57:04.792 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:57:04.798 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T06:57:04.805 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T06:57:04.812 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T06:57:04.812 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:57:04.839 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:57:04.841 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:57:04.864 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T06:57:04.870 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T06:57:04.870 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:57:04.891 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:57:04.902 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:57:04.905 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:57:04.915 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:57:04.918 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:57:04.924 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:57:04.926 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:57:04.932 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:57:04.934 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T06:57:04.941 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T06:57:04.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:57:04.959 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:57:04.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:57:04.991 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T06:57:05.000 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T06:57:05.000 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:57:05.019 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:57:05.034 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:57:05.040 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:57:05.070 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:57:05.118 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T06:57:05.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:57:05.195 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:57:05.201 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:57:05.249 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T06:57:05.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:57:05.308 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:57:05.314 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:57:05.362 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T06:57:05.362 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:57:05.432 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-09-14T06:57:05.435 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-14T06:57:05.437 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-09-14T06:57:05.504 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T06:57:05.512 INFO:teuthology.task.internal:Starting timer... 2024-09-14T06:57:05.512 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T06:57:05.522 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T06:57:05.531 DEBUG:teuthology.task.selinux:Excluding smithi005: OS 'ubuntu' does not support SELinux 2024-09-14T06:57:05.531 DEBUG:teuthology.task.selinux:Excluding smithi110: OS 'ubuntu' does not support SELinux 2024-09-14T06:57:05.531 DEBUG:teuthology.task.selinux:Excluding smithi145: OS 'ubuntu' does not support SELinux 2024-09-14T06:57:05.531 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T06:57:05.531 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-14T06:57:05.531 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T06:57:05.531 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T06:57:05.539 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-14T06:57:05.542 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-14T06:57:05.666 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T06:57:05.677 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T06:57:05.678 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi110.front.sepia.ceph.com,smithi145.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T07:04:45.191 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi145.front.sepia.ceph.com')] 2024-09-14T07:04:45.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T07:04:45.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T07:04:45.291 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-14T07:04:45.373 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-14T07:04:45.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-14T07:04:45.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T07:04:45.458 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-14T07:04:45.532 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-14T07:04:45.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-14T07:04:45.533 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T07:04:45.614 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-09-14T07:04:45.693 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-09-14T07:04:45.694 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T07:04:45.706 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T07:04:45.706 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T07:04:45.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:04:45.709 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T07:04:45.709 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:04:45.712 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T07:04:45.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Command line: ntpd -gq 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: ---------------------------------------------------- 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: corporation. Support and training for ntp-4 are 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: available at https://www.nwtime.org/support 2024-09-14T07:04:45.739 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: ---------------------------------------------------- 2024-09-14T07:04:45.741 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: proto: precision = 0.070 usec (-24) 2024-09-14T07:04:45.741 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: basedate set to 2022-02-04 2024-09-14T07:04:45.741 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: gps base set to 2022-02-06 (week 2196) 2024-09-14T07:04:45.741 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Command line: ntpd -gq 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: ---------------------------------------------------- 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: corporation. Support and training for ntp-4 are 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: available at https://www.nwtime.org/support 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: ---------------------------------------------------- 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: proto: precision = 0.044 usec (-24) 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: basedate set to 2022-02-04 2024-09-14T07:04:45.742 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: gps base set to 2022-02-06 (week 2196) 2024-09-14T07:04:45.743 INFO:teuthology.orchestra.run.smithi005.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T07:04:45.743 INFO:teuthology.orchestra.run.smithi110.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T07:04:45.743 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T07:04:45.743 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T07:04:45.744 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T07:04:45.744 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T07:04:45.744 INFO:teuthology.orchestra.run.smithi005.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T07:04:45.744 INFO:teuthology.orchestra.run.smithi110.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T07:04:45.746 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T07:04:45.746 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T07:04:45.746 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T07:04:45.746 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Listen normally on 3 ens1f0 172.21.15.5:123 2024-09-14T07:04:45.746 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Listen normally on 4 lo [::1]:123 2024-09-14T07:04:45.746 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1442%4]:123 2024-09-14T07:04:45.746 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:45 ntpd[18395]: Listening on routing socket on fd #22 for interface updates 2024-09-14T07:04:45.747 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T07:04:45.747 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T07:04:45.747 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T07:04:45.747 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Listen normally on 3 enp3s0f1 172.21.15.110:123 2024-09-14T07:04:45.747 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Listen normally on 4 lo [::1]:123 2024-09-14T07:04:45.747 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc35%5]:123 2024-09-14T07:04:45.747 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:45 ntpd[18457]: Listening on routing socket on fd #22 for interface updates 2024-09-14T07:04:45.765 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-14T07:04:45.765 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Command line: ntpd -gq 2024-09-14T07:04:45.765 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: ---------------------------------------------------- 2024-09-14T07:04:45.766 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: ntp-4 is maintained by Network Time Foundation, 2024-09-14T07:04:45.766 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-14T07:04:45.766 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: corporation. Support and training for ntp-4 are 2024-09-14T07:04:45.766 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: available at https://www.nwtime.org/support 2024-09-14T07:04:45.766 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: ---------------------------------------------------- 2024-09-14T07:04:45.766 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: proto: precision = 0.043 usec (-24) 2024-09-14T07:04:45.767 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: basedate set to 2022-02-04 2024-09-14T07:04:45.767 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: gps base set to 2022-02-06 (week 2196) 2024-09-14T07:04:45.767 INFO:teuthology.orchestra.run.smithi145.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T07:04:45.768 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-14T07:04:45.768 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: restrict ::: KOD does nothing without LIMITED. 2024-09-14T07:04:45.768 INFO:teuthology.orchestra.run.smithi145.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-14T07:04:45.769 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Listen and drop on 0 v6wildcard [::]:123 2024-09-14T07:04:45.769 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-14T07:04:45.770 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Listen normally on 2 lo 127.0.0.1:123 2024-09-14T07:04:45.770 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Listen normally on 3 enp3s0f1 172.21.15.145:123 2024-09-14T07:04:45.770 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Listen normally on 4 lo [::1]:123 2024-09-14T07:04:45.770 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7791%5]:123 2024-09-14T07:04:45.770 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:45 ntpd[18491]: Listening on routing socket on fd #22 for interface updates 2024-09-14T07:04:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:46 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:46 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:46 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:46 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:46 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:46 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:46.741 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:46 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.741 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:46 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.741 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:46 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:46 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:46 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:46.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:46 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:46.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:46.767 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:46.767 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:46 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:47.742 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:47.742 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:47 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:47.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:47 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:47.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:47 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:48 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:48 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:48 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:48 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:48 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:48 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:48.741 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:48.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:48 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:48.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:48.767 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:48 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:49.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:49.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:49 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:49.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:49 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:49 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:49 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:49 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:49 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:49.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:49 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:49.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:49 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:49 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:49 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:49.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:49 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:49.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:49 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:49.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:49 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:50 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:50 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:50 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:50 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:50 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:50.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:50 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:50.741 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.741 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.741 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:50.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:50 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:50.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:50 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:51.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:51.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:51 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:51.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:51 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:51 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:51 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:51 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:51 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:51.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:51 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:51.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:51 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:51 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:51 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:51.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:51 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:51.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:51 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:51.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:51 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:52.740 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:52.741 INFO:teuthology.orchestra.run.smithi005.stdout:14 Sep 07:04:52 ntpd[18395]: ntpd: time slew -0.001199 s 2024-09-14T07:04:52.741 INFO:teuthology.orchestra.run.smithi005.stdout:ntpd: time slew -0.001199s 2024-09-14T07:04:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T07:04:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:14 Sep 07:04:52 ntpd[18395]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T07:04:52.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.742 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:52.743 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.744 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.744 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:52.744 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:52.744 INFO:teuthology.orchestra.run.smithi110.stdout:14 Sep 07:04:52 ntpd[18457]: ntpd: time slew -0.000822 s 2024-09-14T07:04:52.744 INFO:teuthology.orchestra.run.smithi110.stdout:ntpd: time slew -0.000822s 2024-09-14T07:04:52.745 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T07:04:52.745 INFO:teuthology.orchestra.run.smithi110.stderr:14 Sep 07:04:52 ntpd[18457]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T07:04:52.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.765 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-14T07:04:52.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: can't open /var/log/ntpstats/rawstats.20240914: Permission denied 2024-09-14T07:04:52.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-14T07:04:52.766 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: can't open /var/log/ntpstats/peerstats.20240914: Permission denied 2024-09-14T07:04:52.766 INFO:teuthology.orchestra.run.smithi145.stdout:14 Sep 07:04:52 ntpd[18491]: ntpd: time slew -0.000833 s 2024-09-14T07:04:52.766 INFO:teuthology.orchestra.run.smithi145.stdout:ntpd: time slew -0.000833s 2024-09-14T07:04:52.767 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-14T07:04:52.767 INFO:teuthology.orchestra.run.smithi145.stderr:14 Sep 07:04:52 ntpd[18491]: can't open /var/log/ntpstats/loopstats.20240914: Permission denied 2024-09-14T07:04:52.788 INFO:teuthology.orchestra.run.smithi005.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T07:04:52.788 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================== 2024-09-14T07:04:52.788 INFO:teuthology.orchestra.run.smithi005.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.788 INFO:teuthology.orchestra.run.smithi005.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.788 INFO:teuthology.orchestra.run.smithi005.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.788 INFO:teuthology.orchestra.run.smithi005.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T07:04:52.797 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================== 2024-09-14T07:04:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.823 INFO:teuthology.orchestra.run.smithi145.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T07:04:52.823 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================== 2024-09-14T07:04:52.823 INFO:teuthology.orchestra.run.smithi145.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.823 INFO:teuthology.orchestra.run.smithi145.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.823 INFO:teuthology.orchestra.run.smithi145.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.824 INFO:teuthology.orchestra.run.smithi145.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:04:52.824 INFO:teuthology.run_tasks:Running task install... 2024-09-14T07:04:52.836 DEBUG:teuthology.task.install:project ceph 2024-09-14T07:04:52.836 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}} 2024-09-14T07:04:52.836 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'} 2024-09-14T07:04:52.836 INFO:teuthology.task.install:Using flavor: default 2024-09-14T07:04:52.842 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T07:04:52.842 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-14T07:04:52.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-key list | grep Ceph 2024-09-14T07:04:52.843 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-key list | grep Ceph 2024-09-14T07:04:52.844 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-key list | grep Ceph 2024-09-14T07:04:52.902 INFO:teuthology.orchestra.run.smithi110.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T07:04:52.903 INFO:teuthology.orchestra.run.smithi005.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T07:04:52.921 INFO:teuthology.orchestra.run.smithi145.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-14T07:04:52.928 INFO:teuthology.orchestra.run.smithi110.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T07:04:52.928 INFO:teuthology.orchestra.run.smithi110.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T07:04:52.929 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-14T07:04:52.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T07:04:52.934 INFO:teuthology.orchestra.run.smithi005.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T07:04:52.934 INFO:teuthology.orchestra.run.smithi005.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T07:04:52.934 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-14T07:04:52.935 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T07:04:52.949 INFO:teuthology.orchestra.run.smithi145.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-14T07:04:52.949 INFO:teuthology.orchestra.run.smithi145.stdout:uid [ unknown] Ceph.com (release key) 2024-09-14T07:04:52.950 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-nbd on remote deb x86_64 2024-09-14T07:04:52.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T07:04:53.095 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default/ 2024-09-14T07:04:53.095 INFO:teuthology.task.install.deb:Package version is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:04:53.114 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default/ 2024-09-14T07:04:53.114 INFO:teuthology.task.install.deb:Package version is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:04:53.118 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default/ 2024-09-14T07:04:53.118 INFO:teuthology.task.install.deb:Package version is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:04:53.209 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:04:53.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T07:04:53.222 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:04:53.222 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T07:04:53.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get update 2024-09-14T07:04:53.231 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:04:53.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-14T07:04:53.236 DEBUG:teuthology.orchestra.run.smithi145:> sudo apt-get update 2024-09-14T07:04:53.245 DEBUG:teuthology.orchestra.run.smithi110:> sudo apt-get update 2024-09-14T07:04:53.403 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T07:04:53.412 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T07:04:53.453 INFO:teuthology.orchestra.run.smithi005.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T07:04:53.457 INFO:teuthology.orchestra.run.smithi145.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T07:04:53.466 INFO:teuthology.orchestra.run.smithi110.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy InRelease 2024-09-14T07:04:53.474 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-14T07:04:53.479 INFO:teuthology.orchestra.run.smithi145.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-14T07:04:53.487 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-14T07:04:53.497 INFO:teuthology.orchestra.run.smithi005.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T07:04:53.500 INFO:teuthology.orchestra.run.smithi145.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T07:04:53.509 INFO:teuthology.orchestra.run.smithi110.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-14T07:04:53.518 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-14T07:04:53.521 INFO:teuthology.orchestra.run.smithi145.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-14T07:04:53.530 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T07:04:53.530 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-14T07:04:53.536 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T07:04:53.537 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-14T07:04:53.538 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-14T07:04:53.551 INFO:teuthology.orchestra.run.smithi145.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-14T07:04:53.557 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-14T07:04:53.558 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-14T07:04:53.600 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T07:04:53.612 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T07:04:53.615 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-14T07:04:53.682 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T07:04:53.689 INFO:teuthology.orchestra.run.smithi145.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T07:04:53.696 INFO:teuthology.orchestra.run.smithi110.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-14T07:04:53.830 INFO:teuthology.orchestra.run.smithi145.stdout:Fetched 29.3 kB in 0s (71.0 kB/s) 2024-09-14T07:04:53.832 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 29.3 kB in 0s (71.5 kB/s) 2024-09-14T07:04:53.933 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 29.3 kB in 1s (56.7 kB/s) 2024-09-14T07:04:54.783 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-14T07:04:54.799 DEBUG:teuthology.orchestra.run.smithi110:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4775-g7f24e589-1jammy cephadm=19.3.0-4775-g7f24e589-1jammy ceph-mds=19.3.0-4775-g7f24e589-1jammy ceph-mgr=19.3.0-4775-g7f24e589-1jammy ceph-common=19.3.0-4775-g7f24e589-1jammy ceph-fuse=19.3.0-4775-g7f24e589-1jammy ceph-test=19.3.0-4775-g7f24e589-1jammy ceph-volume=19.3.0-4775-g7f24e589-1jammy radosgw=19.3.0-4775-g7f24e589-1jammy python3-rados=19.3.0-4775-g7f24e589-1jammy python3-rgw=19.3.0-4775-g7f24e589-1jammy python3-cephfs=19.3.0-4775-g7f24e589-1jammy python3-rbd=19.3.0-4775-g7f24e589-1jammy libcephfs2=19.3.0-4775-g7f24e589-1jammy libcephfs-dev=19.3.0-4775-g7f24e589-1jammy librados2=19.3.0-4775-g7f24e589-1jammy librbd1=19.3.0-4775-g7f24e589-1jammy rbd-fuse=19.3.0-4775-g7f24e589-1jammy rbd-nbd=19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:04:54.811 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-14T07:04:54.828 DEBUG:teuthology.orchestra.run.smithi145:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4775-g7f24e589-1jammy cephadm=19.3.0-4775-g7f24e589-1jammy ceph-mds=19.3.0-4775-g7f24e589-1jammy ceph-mgr=19.3.0-4775-g7f24e589-1jammy ceph-common=19.3.0-4775-g7f24e589-1jammy ceph-fuse=19.3.0-4775-g7f24e589-1jammy ceph-test=19.3.0-4775-g7f24e589-1jammy ceph-volume=19.3.0-4775-g7f24e589-1jammy radosgw=19.3.0-4775-g7f24e589-1jammy python3-rados=19.3.0-4775-g7f24e589-1jammy python3-rgw=19.3.0-4775-g7f24e589-1jammy python3-cephfs=19.3.0-4775-g7f24e589-1jammy python3-rbd=19.3.0-4775-g7f24e589-1jammy libcephfs2=19.3.0-4775-g7f24e589-1jammy libcephfs-dev=19.3.0-4775-g7f24e589-1jammy librados2=19.3.0-4775-g7f24e589-1jammy librbd1=19.3.0-4775-g7f24e589-1jammy rbd-fuse=19.3.0-4775-g7f24e589-1jammy rbd-nbd=19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:04:54.866 INFO:teuthology.orchestra.run.smithi110.stdout:Reading package lists... 2024-09-14T07:04:54.889 INFO:teuthology.orchestra.run.smithi145.stdout:Reading package lists... 2024-09-14T07:04:54.971 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-09-14T07:04:54.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4775-g7f24e589-1jammy cephadm=19.3.0-4775-g7f24e589-1jammy ceph-mds=19.3.0-4775-g7f24e589-1jammy ceph-mgr=19.3.0-4775-g7f24e589-1jammy ceph-common=19.3.0-4775-g7f24e589-1jammy ceph-fuse=19.3.0-4775-g7f24e589-1jammy ceph-test=19.3.0-4775-g7f24e589-1jammy ceph-volume=19.3.0-4775-g7f24e589-1jammy radosgw=19.3.0-4775-g7f24e589-1jammy python3-rados=19.3.0-4775-g7f24e589-1jammy python3-rgw=19.3.0-4775-g7f24e589-1jammy python3-cephfs=19.3.0-4775-g7f24e589-1jammy python3-rbd=19.3.0-4775-g7f24e589-1jammy libcephfs2=19.3.0-4775-g7f24e589-1jammy libcephfs-dev=19.3.0-4775-g7f24e589-1jammy librados2=19.3.0-4775-g7f24e589-1jammy librbd1=19.3.0-4775-g7f24e589-1jammy rbd-fuse=19.3.0-4775-g7f24e589-1jammy rbd-nbd=19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:04:55.054 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-09-14T07:04:55.081 INFO:teuthology.orchestra.run.smithi110.stdout:Building dependency tree... 2024-09-14T07:04:55.082 INFO:teuthology.orchestra.run.smithi110.stdout:Reading state information... 2024-09-14T07:04:55.099 INFO:teuthology.orchestra.run.smithi145.stdout:Building dependency tree... 2024-09-14T07:04:55.100 INFO:teuthology.orchestra.run.smithi145.stdout:Reading state information... 2024-09-14T07:04:55.267 INFO:teuthology.orchestra.run.smithi005.stdout:Building dependency tree... 2024-09-14T07:04:55.268 INFO:teuthology.orchestra.run.smithi005.stdout:Reading state information... 2024-09-14T07:04:55.290 INFO:teuthology.orchestra.run.smithi110.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T07:04:55.290 INFO:teuthology.orchestra.run.smithi110.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T07:04:55.290 INFO:teuthology.orchestra.run.smithi110.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T07:04:55.290 INFO:teuthology.orchestra.run.smithi110.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T07:04:55.291 INFO:teuthology.orchestra.run.smithi110.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T07:04:55.291 INFO:teuthology.orchestra.run.smithi110.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T07:04:55.291 INFO:teuthology.orchestra.run.smithi110.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T07:04:55.292 INFO:teuthology.orchestra.run.smithi110.stdout:The following additional packages will be installed: 2024-09-14T07:04:55.292 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T07:04:55.292 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T07:04:55.292 INFO:teuthology.orchestra.run.smithi110.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-14T07:04:55.292 INFO:teuthology.orchestra.run.smithi110.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-14T07:04:55.292 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-14T07:04:55.294 INFO:teuthology.orchestra.run.smithi110.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-14T07:04:55.294 INFO:teuthology.orchestra.run.smithi110.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-14T07:04:55.294 INFO:teuthology.orchestra.run.smithi110.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-14T07:04:55.294 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-14T07:04:55.295 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet zip 2024-09-14T07:04:55.296 INFO:teuthology.orchestra.run.smithi110.stdout:Suggested packages: 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout:Recommended packages: 2024-09-14T07:04:55.297 INFO:teuthology.orchestra.run.smithi110.stdout: btrfs-tools 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout:The following additional packages will be installed: 2024-09-14T07:04:55.301 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-14T07:04:55.302 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet zip 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout:Suggested packages: 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T07:04:55.303 INFO:teuthology.orchestra.run.smithi145.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T07:04:55.304 INFO:teuthology.orchestra.run.smithi145.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T07:04:55.304 INFO:teuthology.orchestra.run.smithi145.stdout:Recommended packages: 2024-09-14T07:04:55.304 INFO:teuthology.orchestra.run.smithi145.stdout: btrfs-tools 2024-09-14T07:04:55.399 INFO:teuthology.orchestra.run.smithi110.stdout:The following NEW packages will be installed: 2024-09-14T07:04:55.399 INFO:teuthology.orchestra.run.smithi110.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T07:04:55.400 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T07:04:55.400 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T07:04:55.400 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T07:04:55.400 INFO:teuthology.orchestra.run.smithi110.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-14T07:04:55.400 INFO:teuthology.orchestra.run.smithi110.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T07:04:55.401 INFO:teuthology.orchestra.run.smithi110.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T07:04:55.401 INFO:teuthology.orchestra.run.smithi110.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T07:04:55.401 INFO:teuthology.orchestra.run.smithi110.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T07:04:55.401 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T07:04:55.401 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T07:04:55.401 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T07:04:55.402 INFO:teuthology.orchestra.run.smithi110.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-14T07:04:55.404 INFO:teuthology.orchestra.run.smithi145.stdout:The following NEW packages will be installed: 2024-09-14T07:04:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T07:04:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T07:04:55.405 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T07:04:55.407 INFO:teuthology.orchestra.run.smithi145.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T07:04:55.408 INFO:teuthology.orchestra.run.smithi145.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-14T07:04:55.468 INFO:teuthology.orchestra.run.smithi145.stdout:0 upgraded, 120 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T07:04:55.468 INFO:teuthology.orchestra.run.smithi145.stdout:Need to get 251 MB of archives. 2024-09-14T07:04:55.469 INFO:teuthology.orchestra.run.smithi145.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-14T07:04:55.469 INFO:teuthology.orchestra.run.smithi145.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T07:04:55.488 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages were automatically installed and are no longer required: 2024-09-14T07:04:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-14T07:04:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-14T07:04:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-14T07:04:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-14T07:04:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-14T07:04:55.491 INFO:teuthology.orchestra.run.smithi005.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-14T07:04:55.491 INFO:teuthology.orchestra.run.smithi005.stdout:The following additional packages will be installed: 2024-09-14T07:04:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-14T07:04:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-14T07:04:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-14T07:04:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet zip 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout:Suggested packages: 2024-09-14T07:04:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout:Recommended packages: 2024-09-14T07:04:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: btrfs-tools 2024-09-14T07:04:55.542 INFO:teuthology.orchestra.run.smithi110.stdout:0 upgraded, 120 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T07:04:55.542 INFO:teuthology.orchestra.run.smithi110.stdout:Need to get 251 MB of archives. 2024-09-14T07:04:55.542 INFO:teuthology.orchestra.run.smithi110.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-14T07:04:55.542 INFO:teuthology.orchestra.run.smithi110.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4775-g7f24e589-1jammy [3,631 kB] 2024-09-14T07:04:55.553 INFO:teuthology.orchestra.run.smithi145.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4775-g7f24e589-1jammy [3,631 kB] 2024-09-14T07:04:55.556 INFO:teuthology.orchestra.run.smithi145.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T07:04:55.559 INFO:teuthology.orchestra.run.smithi145.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T07:04:55.575 INFO:teuthology.orchestra.run.smithi145.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T07:04:55.588 INFO:teuthology.orchestra.run.smithi110.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T07:04:55.603 INFO:teuthology.orchestra.run.smithi005.stdout:The following NEW packages will be installed: 2024-09-14T07:04:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-14T07:04:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-14T07:04:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-14T07:04:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-14T07:04:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-14T07:04:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-14T07:04:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-14T07:04:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-14T07:04:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-14T07:04:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-14T07:04:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-14T07:04:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-14T07:04:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-14T07:04:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: radosgw rbd-fuse rbd-nbd socat unzip xmlstarlet zip 2024-09-14T07:04:55.634 INFO:teuthology.orchestra.run.smithi145.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T07:04:55.638 INFO:teuthology.orchestra.run.smithi145.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T07:04:55.650 INFO:teuthology.orchestra.run.smithi145.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T07:04:55.655 INFO:teuthology.orchestra.run.smithi145.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-14T07:04:55.656 INFO:teuthology.orchestra.run.smithi145.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T07:04:55.657 INFO:teuthology.orchestra.run.smithi145.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T07:04:55.657 INFO:teuthology.orchestra.run.smithi145.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T07:04:55.665 INFO:teuthology.orchestra.run.smithi005.stdout:0 upgraded, 120 newly installed, 0 to remove and 331 not upgraded. 2024-09-14T07:04:55.665 INFO:teuthology.orchestra.run.smithi005.stdout:Need to get 251 MB of archives. 2024-09-14T07:04:55.665 INFO:teuthology.orchestra.run.smithi005.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-14T07:04:55.666 INFO:teuthology.orchestra.run.smithi005.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-14T07:04:55.666 INFO:teuthology.orchestra.run.smithi145.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T07:04:55.668 INFO:teuthology.orchestra.run.smithi145.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T07:04:55.670 INFO:teuthology.orchestra.run.smithi145.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T07:04:55.672 INFO:teuthology.orchestra.run.smithi145.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T07:04:55.672 INFO:teuthology.orchestra.run.smithi145.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T07:04:55.689 INFO:teuthology.orchestra.run.smithi145.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T07:04:55.692 INFO:teuthology.orchestra.run.smithi145.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T07:04:55.694 INFO:teuthology.orchestra.run.smithi145.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T07:04:55.695 INFO:teuthology.orchestra.run.smithi145.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T07:04:55.696 INFO:teuthology.orchestra.run.smithi145.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T07:04:55.697 INFO:teuthology.orchestra.run.smithi145.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T07:04:55.697 INFO:teuthology.orchestra.run.smithi145.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T07:04:55.698 INFO:teuthology.orchestra.run.smithi145.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T07:04:55.706 INFO:teuthology.orchestra.run.smithi145.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T07:04:55.706 INFO:teuthology.orchestra.run.smithi145.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T07:04:55.723 INFO:teuthology.orchestra.run.smithi145.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T07:04:55.723 INFO:teuthology.orchestra.run.smithi145.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T07:04:55.726 INFO:teuthology.orchestra.run.smithi145.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T07:04:55.727 INFO:teuthology.orchestra.run.smithi145.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T07:04:55.728 INFO:teuthology.orchestra.run.smithi145.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T07:04:55.728 INFO:teuthology.orchestra.run.smithi145.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T07:04:55.730 INFO:teuthology.orchestra.run.smithi145.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T07:04:55.730 INFO:teuthology.orchestra.run.smithi145.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T07:04:55.739 INFO:teuthology.orchestra.run.smithi145.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T07:04:55.741 INFO:teuthology.orchestra.run.smithi145.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T07:04:55.753 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T07:04:55.756 INFO:teuthology.orchestra.run.smithi005.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T07:04:55.756 INFO:teuthology.orchestra.run.smithi145.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T07:04:55.757 INFO:teuthology.orchestra.run.smithi145.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T07:04:55.762 INFO:teuthology.orchestra.run.smithi110.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4775-g7f24e589-1jammy [3,330 kB] 2024-09-14T07:04:55.764 INFO:teuthology.orchestra.run.smithi145.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T07:04:55.764 INFO:teuthology.orchestra.run.smithi145.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T07:04:55.765 INFO:teuthology.orchestra.run.smithi145.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T07:04:55.766 INFO:teuthology.orchestra.run.smithi145.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T07:04:55.767 INFO:teuthology.orchestra.run.smithi145.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T07:04:55.770 INFO:teuthology.orchestra.run.smithi145.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T07:04:55.771 INFO:teuthology.orchestra.run.smithi145.stdout:Get:46 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4775-g7f24e589-1jammy [3,330 kB] 2024-09-14T07:04:55.772 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T07:04:55.773 INFO:teuthology.orchestra.run.smithi145.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T07:04:55.775 INFO:teuthology.orchestra.run.smithi145.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T07:04:55.788 INFO:teuthology.orchestra.run.smithi145.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T07:04:55.790 INFO:teuthology.orchestra.run.smithi005.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4775-g7f24e589-1jammy [3,631 kB] 2024-09-14T07:04:55.791 INFO:teuthology.orchestra.run.smithi145.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T07:04:55.792 INFO:teuthology.orchestra.run.smithi145.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T07:04:55.793 INFO:teuthology.orchestra.run.smithi145.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T07:04:55.796 INFO:teuthology.orchestra.run.smithi145.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T07:04:55.831 INFO:teuthology.orchestra.run.smithi005.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T07:04:55.836 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T07:04:55.849 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T07:04:55.851 INFO:teuthology.orchestra.run.smithi110.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4775-g7f24e589-1jammy [722 kB] 2024-09-14T07:04:55.853 INFO:teuthology.orchestra.run.smithi005.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-14T07:04:55.855 INFO:teuthology.orchestra.run.smithi005.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T07:04:55.855 INFO:teuthology.orchestra.run.smithi005.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T07:04:55.856 INFO:teuthology.orchestra.run.smithi145.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T07:04:55.856 INFO:teuthology.orchestra.run.smithi005.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T07:04:55.856 INFO:teuthology.orchestra.run.smithi145.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T07:04:55.858 INFO:teuthology.orchestra.run.smithi145.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T07:04:55.858 INFO:teuthology.orchestra.run.smithi145.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T07:04:55.859 INFO:teuthology.orchestra.run.smithi145.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T07:04:55.865 INFO:teuthology.orchestra.run.smithi005.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T07:04:55.867 INFO:teuthology.orchestra.run.smithi005.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T07:04:55.870 INFO:teuthology.orchestra.run.smithi005.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T07:04:55.871 INFO:teuthology.orchestra.run.smithi005.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T07:04:55.871 INFO:teuthology.orchestra.run.smithi005.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T07:04:55.874 INFO:teuthology.orchestra.run.smithi145.stdout:Get:59 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4775-g7f24e589-1jammy [722 kB] 2024-09-14T07:04:55.876 INFO:teuthology.orchestra.run.smithi145.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T07:04:55.878 INFO:teuthology.orchestra.run.smithi145.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T07:04:55.878 INFO:teuthology.orchestra.run.smithi145.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T07:04:55.882 INFO:teuthology.orchestra.run.smithi110.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4775-g7f24e589-1jammy [356 kB] 2024-09-14T07:04:55.887 INFO:teuthology.orchestra.run.smithi005.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T07:04:55.889 INFO:teuthology.orchestra.run.smithi005.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T07:04:55.890 INFO:teuthology.orchestra.run.smithi110.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4775-g7f24e589-1jammy [32.8 kB] 2024-09-14T07:04:55.890 INFO:teuthology.orchestra.run.smithi110.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4775-g7f24e589-1jammy [180 kB] 2024-09-14T07:04:55.891 INFO:teuthology.orchestra.run.smithi005.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T07:04:55.892 INFO:teuthology.orchestra.run.smithi005.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T07:04:55.892 INFO:teuthology.orchestra.run.smithi005.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T07:04:55.893 INFO:teuthology.orchestra.run.smithi145.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T07:04:55.893 INFO:teuthology.orchestra.run.smithi145.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T07:04:55.893 INFO:teuthology.orchestra.run.smithi145.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T07:04:55.894 INFO:teuthology.orchestra.run.smithi110.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4775-g7f24e589-1jammy [71.2 kB] 2024-09-14T07:04:55.894 INFO:teuthology.orchestra.run.smithi145.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T07:04:55.895 INFO:teuthology.orchestra.run.smithi145.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T07:04:55.895 INFO:teuthology.orchestra.run.smithi145.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T07:04:55.896 INFO:teuthology.orchestra.run.smithi110.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4775-g7f24e589-1jammy [338 kB] 2024-09-14T07:04:55.902 INFO:teuthology.orchestra.run.smithi110.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4775-g7f24e589-1jammy [7,984 kB] 2024-09-14T07:04:55.904 INFO:teuthology.orchestra.run.smithi005.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T07:04:55.904 INFO:teuthology.orchestra.run.smithi005.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T07:04:55.905 INFO:teuthology.orchestra.run.smithi005.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T07:04:55.905 INFO:teuthology.orchestra.run.smithi005.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T07:04:55.905 INFO:teuthology.orchestra.run.smithi005.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T07:04:55.906 INFO:teuthology.orchestra.run.smithi145.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4775-g7f24e589-1jammy [356 kB] 2024-09-14T07:04:55.909 INFO:teuthology.orchestra.run.smithi145.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T07:04:55.910 INFO:teuthology.orchestra.run.smithi145.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T07:04:55.913 INFO:teuthology.orchestra.run.smithi145.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4775-g7f24e589-1jammy [32.8 kB] 2024-09-14T07:04:55.914 INFO:teuthology.orchestra.run.smithi145.stdout:Get:73 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4775-g7f24e589-1jammy [180 kB] 2024-09-14T07:04:55.918 INFO:teuthology.orchestra.run.smithi145.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T07:04:55.918 INFO:teuthology.orchestra.run.smithi145.stdout:Get:75 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4775-g7f24e589-1jammy [71.2 kB] 2024-09-14T07:04:55.920 INFO:teuthology.orchestra.run.smithi145.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4775-g7f24e589-1jammy [338 kB] 2024-09-14T07:04:55.921 INFO:teuthology.orchestra.run.smithi005.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T07:04:55.921 INFO:teuthology.orchestra.run.smithi005.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T07:04:55.924 INFO:teuthology.orchestra.run.smithi005.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T07:04:55.924 INFO:teuthology.orchestra.run.smithi005.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T07:04:55.925 INFO:teuthology.orchestra.run.smithi005.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T07:04:55.926 INFO:teuthology.orchestra.run.smithi145.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T07:04:55.927 INFO:teuthology.orchestra.run.smithi145.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T07:04:55.928 INFO:teuthology.orchestra.run.smithi145.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T07:04:55.931 INFO:teuthology.orchestra.run.smithi145.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T07:04:55.934 INFO:teuthology.orchestra.run.smithi145.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T07:04:55.935 INFO:teuthology.orchestra.run.smithi145.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T07:04:55.935 INFO:teuthology.orchestra.run.smithi145.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T07:04:55.938 INFO:teuthology.orchestra.run.smithi005.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T07:04:55.939 INFO:teuthology.orchestra.run.smithi005.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T07:04:55.939 INFO:teuthology.orchestra.run.smithi005.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T07:04:55.939 INFO:teuthology.orchestra.run.smithi005.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T07:04:55.941 INFO:teuthology.orchestra.run.smithi005.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T07:04:55.942 INFO:teuthology.orchestra.run.smithi145.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T07:04:55.943 INFO:teuthology.orchestra.run.smithi145.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T07:04:55.944 INFO:teuthology.orchestra.run.smithi145.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4775-g7f24e589-1jammy [7,984 kB] 2024-09-14T07:04:55.955 INFO:teuthology.orchestra.run.smithi005.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T07:04:55.955 INFO:teuthology.orchestra.run.smithi005.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T07:04:55.959 INFO:teuthology.orchestra.run.smithi145.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T07:04:55.960 INFO:teuthology.orchestra.run.smithi145.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T07:04:55.960 INFO:teuthology.orchestra.run.smithi145.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T07:04:55.961 INFO:teuthology.orchestra.run.smithi145.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T07:04:55.961 INFO:teuthology.orchestra.run.smithi005.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T07:04:55.961 INFO:teuthology.orchestra.run.smithi005.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T07:04:55.961 INFO:teuthology.orchestra.run.smithi005.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T07:04:55.962 INFO:teuthology.orchestra.run.smithi145.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T07:04:55.963 INFO:teuthology.orchestra.run.smithi145.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T07:04:55.969 INFO:teuthology.orchestra.run.smithi145.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T07:04:55.969 INFO:teuthology.orchestra.run.smithi145.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T07:04:55.971 INFO:teuthology.orchestra.run.smithi005.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T07:04:55.973 INFO:teuthology.orchestra.run.smithi005.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T07:04:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T07:04:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T07:04:55.976 INFO:teuthology.orchestra.run.smithi145.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T07:04:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T07:04:55.978 INFO:teuthology.orchestra.run.smithi145.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T07:04:55.989 INFO:teuthology.orchestra.run.smithi005.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T07:04:55.991 INFO:teuthology.orchestra.run.smithi005.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T07:04:55.991 INFO:teuthology.orchestra.run.smithi005.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T07:04:55.993 INFO:teuthology.orchestra.run.smithi005.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T07:04:55.993 INFO:teuthology.orchestra.run.smithi145.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T07:04:55.995 INFO:teuthology.orchestra.run.smithi005.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T07:04:56.012 INFO:teuthology.orchestra.run.smithi110.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-14T07:04:56.024 INFO:teuthology.orchestra.run.smithi110.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-14T07:04:56.048 INFO:teuthology.orchestra.run.smithi005.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T07:04:56.049 INFO:teuthology.orchestra.run.smithi005.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T07:04:56.050 INFO:teuthology.orchestra.run.smithi005.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T07:04:56.050 INFO:teuthology.orchestra.run.smithi005.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T07:04:56.051 INFO:teuthology.orchestra.run.smithi005.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T07:04:56.069 INFO:teuthology.orchestra.run.smithi005.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T07:04:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T07:04:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T07:04:56.086 INFO:teuthology.orchestra.run.smithi005.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T07:04:56.086 INFO:teuthology.orchestra.run.smithi005.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T07:04:56.087 INFO:teuthology.orchestra.run.smithi005.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T07:04:56.087 INFO:teuthology.orchestra.run.smithi005.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T07:04:56.087 INFO:teuthology.orchestra.run.smithi005.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T07:04:56.088 INFO:teuthology.orchestra.run.smithi005.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T07:04:56.101 INFO:teuthology.orchestra.run.smithi005.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T07:04:56.101 INFO:teuthology.orchestra.run.smithi005.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T07:04:56.104 INFO:teuthology.orchestra.run.smithi005.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T07:04:56.105 INFO:teuthology.orchestra.run.smithi110.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-14T07:04:56.118 INFO:teuthology.orchestra.run.smithi005.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T07:04:56.119 INFO:teuthology.orchestra.run.smithi005.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T07:04:56.119 INFO:teuthology.orchestra.run.smithi005.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T07:04:56.123 INFO:teuthology.orchestra.run.smithi005.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T07:04:56.125 INFO:teuthology.orchestra.run.smithi005.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T07:04:56.125 INFO:teuthology.orchestra.run.smithi005.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T07:04:56.126 INFO:teuthology.orchestra.run.smithi005.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T07:04:56.134 INFO:teuthology.orchestra.run.smithi005.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T07:04:56.135 INFO:teuthology.orchestra.run.smithi005.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T07:04:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T07:04:56.152 INFO:teuthology.orchestra.run.smithi005.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T07:04:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T07:04:56.153 INFO:teuthology.orchestra.run.smithi005.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T07:04:56.154 INFO:teuthology.orchestra.run.smithi005.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T07:04:56.155 INFO:teuthology.orchestra.run.smithi005.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T07:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T07:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T07:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T07:04:56.171 INFO:teuthology.orchestra.run.smithi005.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T07:04:56.171 INFO:teuthology.orchestra.run.smithi005.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T07:04:56.192 INFO:teuthology.orchestra.run.smithi110.stdout:Get:14 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4775-g7f24e589-1jammy [112 kB] 2024-09-14T07:04:56.195 INFO:teuthology.orchestra.run.smithi110.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4775-g7f24e589-1jammy [473 kB] 2024-09-14T07:04:56.204 INFO:teuthology.orchestra.run.smithi110.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4775-g7f24e589-1jammy [28.3 MB] 2024-09-14T07:04:56.225 INFO:teuthology.orchestra.run.smithi145.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4775-g7f24e589-1jammy [112 kB] 2024-09-14T07:04:56.228 INFO:teuthology.orchestra.run.smithi145.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4775-g7f24e589-1jammy [473 kB] 2024-09-14T07:04:56.238 INFO:teuthology.orchestra.run.smithi145.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4775-g7f24e589-1jammy [28.3 MB] 2024-09-14T07:04:56.335 INFO:teuthology.orchestra.run.smithi110.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-14T07:04:56.346 INFO:teuthology.orchestra.run.smithi110.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-14T07:04:56.375 INFO:teuthology.orchestra.run.smithi110.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-14T07:04:56.384 INFO:teuthology.orchestra.run.smithi110.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-14T07:04:56.388 INFO:teuthology.orchestra.run.smithi110.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-14T07:04:56.388 INFO:teuthology.orchestra.run.smithi110.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-14T07:04:56.389 INFO:teuthology.orchestra.run.smithi110.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-14T07:04:56.406 INFO:teuthology.orchestra.run.smithi110.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-14T07:04:56.410 INFO:teuthology.orchestra.run.smithi110.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-14T07:04:56.417 INFO:teuthology.orchestra.run.smithi110.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-14T07:04:56.497 INFO:teuthology.orchestra.run.smithi110.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-14T07:04:56.497 INFO:teuthology.orchestra.run.smithi110.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-14T07:04:56.501 INFO:teuthology.orchestra.run.smithi110.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-14T07:04:56.505 INFO:teuthology.orchestra.run.smithi110.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-14T07:04:56.509 INFO:teuthology.orchestra.run.smithi110.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-14T07:04:56.510 INFO:teuthology.orchestra.run.smithi110.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-14T07:04:56.510 INFO:teuthology.orchestra.run.smithi110.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-14T07:04:56.512 INFO:teuthology.orchestra.run.smithi110.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-14T07:04:56.513 INFO:teuthology.orchestra.run.smithi110.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-14T07:04:56.581 INFO:teuthology.orchestra.run.smithi110.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-14T07:04:56.596 INFO:teuthology.orchestra.run.smithi110.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-14T07:04:56.596 INFO:teuthology.orchestra.run.smithi110.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-14T07:04:56.597 INFO:teuthology.orchestra.run.smithi110.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-14T07:04:56.598 INFO:teuthology.orchestra.run.smithi110.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-14T07:04:56.603 INFO:teuthology.orchestra.run.smithi110.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-14T07:04:56.604 INFO:teuthology.orchestra.run.smithi110.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-14T07:04:56.663 INFO:teuthology.orchestra.run.smithi110.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-14T07:04:56.664 INFO:teuthology.orchestra.run.smithi110.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-14T07:04:56.666 INFO:teuthology.orchestra.run.smithi110.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-14T07:04:56.744 INFO:teuthology.orchestra.run.smithi110.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-14T07:04:56.745 INFO:teuthology.orchestra.run.smithi110.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-14T07:04:56.747 INFO:teuthology.orchestra.run.smithi110.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-14T07:04:56.749 INFO:teuthology.orchestra.run.smithi110.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-14T07:04:56.749 INFO:teuthology.orchestra.run.smithi110.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-14T07:04:56.760 INFO:teuthology.orchestra.run.smithi110.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-14T07:04:56.826 INFO:teuthology.orchestra.run.smithi110.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-14T07:04:56.827 INFO:teuthology.orchestra.run.smithi110.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-14T07:04:56.828 INFO:teuthology.orchestra.run.smithi110.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-14T07:04:56.830 INFO:teuthology.orchestra.run.smithi110.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-14T07:04:56.907 INFO:teuthology.orchestra.run.smithi110.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-14T07:04:56.909 INFO:teuthology.orchestra.run.smithi110.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-14T07:04:56.911 INFO:teuthology.orchestra.run.smithi110.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-14T07:04:56.913 INFO:teuthology.orchestra.run.smithi110.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-14T07:04:56.918 INFO:teuthology.orchestra.run.smithi110.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-14T07:04:56.919 INFO:teuthology.orchestra.run.smithi110.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-14T07:04:56.989 INFO:teuthology.orchestra.run.smithi110.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-14T07:04:56.993 INFO:teuthology.orchestra.run.smithi110.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-14T07:04:57.157 INFO:teuthology.orchestra.run.smithi110.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-14T07:04:57.158 INFO:teuthology.orchestra.run.smithi110.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-14T07:04:57.160 INFO:teuthology.orchestra.run.smithi110.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-14T07:04:57.161 INFO:teuthology.orchestra.run.smithi110.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-14T07:04:57.163 INFO:teuthology.orchestra.run.smithi005.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4775-g7f24e589-1jammy [3,330 kB] 2024-09-14T07:04:57.163 INFO:teuthology.orchestra.run.smithi110.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-14T07:04:57.232 INFO:teuthology.orchestra.run.smithi110.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-14T07:04:57.235 INFO:teuthology.orchestra.run.smithi110.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-14T07:04:57.307 INFO:teuthology.orchestra.run.smithi110.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-14T07:04:57.309 INFO:teuthology.orchestra.run.smithi110.stdout:Get:72 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4775-g7f24e589-1jammy [5,206 kB] 2024-09-14T07:04:57.337 INFO:teuthology.orchestra.run.smithi110.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-14T07:04:57.337 INFO:teuthology.orchestra.run.smithi110.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-14T07:04:57.338 INFO:teuthology.orchestra.run.smithi110.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-14T07:04:57.339 INFO:teuthology.orchestra.run.smithi110.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-14T07:04:57.386 INFO:teuthology.orchestra.run.smithi145.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4775-g7f24e589-1jammy [5,206 kB] 2024-09-14T07:04:57.414 INFO:teuthology.orchestra.run.smithi110.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-14T07:04:57.414 INFO:teuthology.orchestra.run.smithi110.stdout:Get:78 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4775-g7f24e589-1jammy [251 kB] 2024-09-14T07:04:57.414 INFO:teuthology.orchestra.run.smithi110.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-14T07:04:57.415 INFO:teuthology.orchestra.run.smithi110.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-14T07:04:57.416 INFO:teuthology.orchestra.run.smithi110.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-14T07:04:57.416 INFO:teuthology.orchestra.run.smithi110.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4775-g7f24e589-1jammy [124 kB] 2024-09-14T07:04:57.418 INFO:teuthology.orchestra.run.smithi110.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4775-g7f24e589-1jammy [1,561 kB] 2024-09-14T07:04:57.422 INFO:teuthology.orchestra.run.smithi110.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-14T07:04:57.433 INFO:teuthology.orchestra.run.smithi110.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4775-g7f24e589-1jammy [6,311 kB] 2024-09-14T07:04:57.469 INFO:teuthology.orchestra.run.smithi145.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4775-g7f24e589-1jammy [251 kB] 2024-09-14T07:04:57.471 INFO:teuthology.orchestra.run.smithi145.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4775-g7f24e589-1jammy [124 kB] 2024-09-14T07:04:57.473 INFO:teuthology.orchestra.run.smithi145.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4775-g7f24e589-1jammy [1,561 kB] 2024-09-14T07:04:57.486 INFO:teuthology.orchestra.run.smithi110.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-14T07:04:57.489 INFO:teuthology.orchestra.run.smithi110.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-14T07:04:57.491 INFO:teuthology.orchestra.run.smithi110.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-14T07:04:57.512 INFO:teuthology.orchestra.run.smithi145.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4775-g7f24e589-1jammy [6,311 kB] 2024-09-14T07:04:57.568 INFO:teuthology.orchestra.run.smithi110.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-14T07:04:57.574 INFO:teuthology.orchestra.run.smithi110.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-14T07:04:57.576 INFO:teuthology.orchestra.run.smithi110.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-14T07:04:57.577 INFO:teuthology.orchestra.run.smithi110.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-14T07:04:57.588 INFO:teuthology.orchestra.run.smithi110.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-14T07:04:57.589 INFO:teuthology.orchestra.run.smithi110.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-14T07:04:57.597 INFO:teuthology.orchestra.run.smithi110.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-14T07:04:57.639 INFO:teuthology.orchestra.run.smithi110.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4775-g7f24e589-1jammy [23.1 MB] 2024-09-14T07:04:57.649 INFO:teuthology.orchestra.run.smithi110.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-14T07:04:57.651 INFO:teuthology.orchestra.run.smithi110.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-14T07:04:57.651 INFO:teuthology.orchestra.run.smithi110.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-14T07:04:57.731 INFO:teuthology.orchestra.run.smithi110.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-14T07:04:57.733 INFO:teuthology.orchestra.run.smithi110.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-14T07:04:57.749 INFO:teuthology.orchestra.run.smithi145.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4775-g7f24e589-1jammy [23.1 MB] 2024-09-14T07:04:57.750 INFO:teuthology.orchestra.run.smithi110.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-14T07:04:57.750 INFO:teuthology.orchestra.run.smithi110.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-14T07:04:57.751 INFO:teuthology.orchestra.run.smithi110.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-14T07:04:57.756 INFO:teuthology.orchestra.run.smithi110.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-14T07:04:57.758 INFO:teuthology.orchestra.run.smithi110.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-14T07:04:57.886 INFO:teuthology.orchestra.run.smithi005.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4775-g7f24e589-1jammy [722 kB] 2024-09-14T07:04:58.070 INFO:teuthology.orchestra.run.smithi005.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4775-g7f24e589-1jammy [356 kB] 2024-09-14T07:04:58.200 INFO:teuthology.orchestra.run.smithi005.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4775-g7f24e589-1jammy [32.8 kB] 2024-09-14T07:04:58.202 INFO:teuthology.orchestra.run.smithi005.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4775-g7f24e589-1jammy [180 kB] 2024-09-14T07:04:58.250 INFO:teuthology.orchestra.run.smithi005.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4775-g7f24e589-1jammy [71.2 kB] 2024-09-14T07:04:58.255 INFO:teuthology.orchestra.run.smithi005.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4775-g7f24e589-1jammy [338 kB] 2024-09-14T07:04:58.350 INFO:teuthology.orchestra.run.smithi005.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4775-g7f24e589-1jammy [7,984 kB] 2024-09-14T07:04:58.525 INFO:teuthology.orchestra.run.smithi110.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4775-g7f24e589-1jammy [14.1 kB] 2024-09-14T07:04:58.525 INFO:teuthology.orchestra.run.smithi110.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4775-g7f24e589-1jammy [880 kB] 2024-09-14T07:04:58.542 INFO:teuthology.orchestra.run.smithi110.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4775-g7f24e589-1jammy [2,452 kB] 2024-09-14T07:04:58.584 INFO:teuthology.orchestra.run.smithi110.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4775-g7f24e589-1jammy [812 kB] 2024-09-14T07:04:58.594 INFO:teuthology.orchestra.run.smithi110.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4775-g7f24e589-1jammy [160 kB] 2024-09-14T07:04:58.596 INFO:teuthology.orchestra.run.smithi110.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4775-g7f24e589-1jammy [61.4 MB] 2024-09-14T07:04:58.648 INFO:teuthology.orchestra.run.smithi145.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4775-g7f24e589-1jammy [14.1 kB] 2024-09-14T07:04:58.649 INFO:teuthology.orchestra.run.smithi145.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4775-g7f24e589-1jammy [880 kB] 2024-09-14T07:04:58.667 INFO:teuthology.orchestra.run.smithi145.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4775-g7f24e589-1jammy [2,452 kB] 2024-09-14T07:04:58.710 INFO:teuthology.orchestra.run.smithi145.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4775-g7f24e589-1jammy [812 kB] 2024-09-14T07:04:58.720 INFO:teuthology.orchestra.run.smithi145.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4775-g7f24e589-1jammy [160 kB] 2024-09-14T07:04:58.736 INFO:teuthology.orchestra.run.smithi145.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4775-g7f24e589-1jammy [61.4 MB] 2024-09-14T07:05:00.806 INFO:teuthology.orchestra.run.smithi005.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4775-g7f24e589-1jammy [112 kB] 2024-09-14T07:05:00.809 INFO:teuthology.orchestra.run.smithi005.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4775-g7f24e589-1jammy [473 kB] 2024-09-14T07:05:00.903 INFO:teuthology.orchestra.run.smithi005.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4775-g7f24e589-1jammy [28.3 MB] 2024-09-14T07:05:00.946 INFO:teuthology.orchestra.run.smithi110.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4775-g7f24e589-1jammy [8,625 kB] 2024-09-14T07:05:01.057 INFO:teuthology.orchestra.run.smithi110.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4775-g7f24e589-1jammy [14.2 kB] 2024-09-14T07:05:01.057 INFO:teuthology.orchestra.run.smithi110.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4775-g7f24e589-1jammy [55.8 MB] 2024-09-14T07:05:01.158 INFO:teuthology.orchestra.run.smithi145.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4775-g7f24e589-1jammy [8,625 kB] 2024-09-14T07:05:01.405 INFO:teuthology.orchestra.run.smithi145.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4775-g7f24e589-1jammy [14.2 kB] 2024-09-14T07:05:01.406 INFO:teuthology.orchestra.run.smithi145.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4775-g7f24e589-1jammy [55.8 MB] 2024-09-14T07:05:03.047 INFO:teuthology.orchestra.run.smithi110.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4775-g7f24e589-1jammy [130 kB] 2024-09-14T07:05:03.117 INFO:teuthology.orchestra.run.smithi110.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4775-g7f24e589-1jammy [39.0 kB] 2024-09-14T07:05:03.118 INFO:teuthology.orchestra.run.smithi110.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4775-g7f24e589-1jammy [15.9 MB] 2024-09-14T07:05:03.370 INFO:teuthology.orchestra.run.smithi145.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4775-g7f24e589-1jammy [130 kB] 2024-09-14T07:05:03.372 INFO:teuthology.orchestra.run.smithi145.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4775-g7f24e589-1jammy [39.0 kB] 2024-09-14T07:05:03.437 INFO:teuthology.orchestra.run.smithi145.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4775-g7f24e589-1jammy [15.9 MB] 2024-09-14T07:05:03.512 INFO:teuthology.orchestra.run.smithi110.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4775-g7f24e589-1jammy [91.3 kB] 2024-09-14T07:05:03.513 INFO:teuthology.orchestra.run.smithi110.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.3.0-4775-g7f24e589-1jammy [178 kB] 2024-09-14T07:05:03.759 INFO:teuthology.orchestra.run.smithi145.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4775-g7f24e589-1jammy [91.3 kB] 2024-09-14T07:05:03.760 INFO:teuthology.orchestra.run.smithi145.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.3.0-4775-g7f24e589-1jammy [178 kB] 2024-09-14T07:05:04.068 INFO:teuthology.orchestra.run.smithi110.stdout:Fetched 251 MB in 8s (31.0 MB/s) 2024-09-14T07:05:04.225 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T07:05:04.325 INFO:teuthology.orchestra.run.smithi110.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T07:05:04.330 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T07:05:04.332 INFO:teuthology.orchestra.run.smithi145.stdout:Fetched 251 MB in 8s (30.1 MB/s) 2024-09-14T07:05:04.415 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T07:05:04.497 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T07:05:04.597 INFO:teuthology.orchestra.run.smithi145.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T07:05:04.603 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T07:05:04.695 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T07:05:04.744 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T07:05:04.770 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T07:05:04.819 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T07:05:05.040 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T07:05:05.067 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T07:05:05.071 INFO:teuthology.orchestra.run.smithi005.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4775-g7f24e589-1jammy [5,206 kB] 2024-09-14T07:05:05.107 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T07:05:05.113 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T07:05:05.139 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T07:05:05.172 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T07:05:05.407 INFO:teuthology.orchestra.run.smithi005.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4775-g7f24e589-1jammy [251 kB] 2024-09-14T07:05:05.425 INFO:teuthology.orchestra.run.smithi005.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4775-g7f24e589-1jammy [124 kB] 2024-09-14T07:05:05.427 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T07:05:05.443 INFO:teuthology.orchestra.run.smithi005.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4775-g7f24e589-1jammy [1,561 kB] 2024-09-14T07:05:05.453 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T07:05:05.493 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T07:05:05.547 INFO:teuthology.orchestra.run.smithi005.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4775-g7f24e589-1jammy [6,311 kB] 2024-09-14T07:05:05.642 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T07:05:05.668 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:05.717 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:05.931 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T07:05:05.957 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:05.962 INFO:teuthology.orchestra.run.smithi005.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4775-g7f24e589-1jammy [23.1 MB] 2024-09-14T07:05:06.005 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:06.095 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T07:05:06.121 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:06.162 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:06.400 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T07:05:06.426 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:06.466 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:06.481 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T07:05:06.507 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:06.548 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:06.778 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T07:05:06.804 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:06.861 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:06.893 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T07:05:06.919 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T07:05:06.959 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T07:05:07.223 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T07:05:07.249 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T07:05:07.289 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T07:05:07.472 INFO:teuthology.orchestra.run.smithi005.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4775-g7f24e589-1jammy [14.1 kB] 2024-09-14T07:05:07.472 INFO:teuthology.orchestra.run.smithi005.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4775-g7f24e589-1jammy [880 kB] 2024-09-14T07:05:07.495 INFO:teuthology.orchestra.run.smithi005.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4775-g7f24e589-1jammy [2,452 kB] 2024-09-14T07:05:07.567 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librados2. 2024-09-14T07:05:07.582 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../007-librados2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:07.617 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:07.623 INFO:teuthology.orchestra.run.smithi005.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4775-g7f24e589-1jammy [812 kB] 2024-09-14T07:05:07.678 INFO:teuthology.orchestra.run.smithi005.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4775-g7f24e589-1jammy [160 kB] 2024-09-14T07:05:07.679 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librados2. 2024-09-14T07:05:07.682 INFO:teuthology.orchestra.run.smithi005.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4775-g7f24e589-1jammy [61.4 MB] 2024-09-14T07:05:07.705 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../007-librados2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:07.737 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:08.095 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libnbd0. 2024-09-14T07:05:08.111 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-14T07:05:08.145 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-14T07:05:08.215 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libnbd0. 2024-09-14T07:05:08.241 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-14T07:05:08.282 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-14T07:05:08.464 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librbd1. 2024-09-14T07:05:08.491 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../009-librbd1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:08.531 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:08.576 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librbd1. 2024-09-14T07:05:08.602 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../009-librbd1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:08.643 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:08.975 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T07:05:09.002 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:09.029 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T07:05:09.042 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:09.044 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:09.079 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:09.286 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rados. 2024-09-14T07:05:09.302 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:09.336 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:09.356 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rados. 2024-09-14T07:05:09.372 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:09.415 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:09.639 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T07:05:09.665 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:09.705 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:09.742 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T07:05:09.769 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:09.809 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:09.941 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T07:05:09.957 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:09.999 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:10.070 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T07:05:10.096 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:10.153 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:10.319 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T07:05:10.346 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:10.385 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:10.406 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T07:05:10.421 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:10.456 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:10.705 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T07:05:10.732 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T07:05:10.767 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T07:05:10.772 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T07:05:10.782 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T07:05:10.817 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T07:05:10.897 INFO:teuthology.orchestra.run.smithi005.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4775-g7f24e589-1jammy [8,625 kB] 2024-09-14T07:05:11.092 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T07:05:11.104 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T07:05:11.109 INFO:teuthology.orchestra.run.smithi005.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4775-g7f24e589-1jammy [14.2 kB] 2024-09-14T07:05:11.110 INFO:teuthology.orchestra.run.smithi005.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4775-g7f24e589-1jammy [55.8 MB] 2024-09-14T07:05:11.118 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T07:05:11.130 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T07:05:11.158 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T07:05:11.170 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T07:05:11.431 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T07:05:11.452 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T07:05:11.457 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:11.468 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:11.498 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:11.511 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:11.793 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T07:05:11.814 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T07:05:11.819 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T07:05:11.840 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T07:05:11.851 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T07:05:11.872 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T07:05:12.124 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T07:05:12.130 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T07:05:12.140 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T07:05:12.145 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T07:05:12.183 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T07:05:12.188 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T07:05:12.517 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T07:05:12.532 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T07:05:12.575 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T07:05:12.753 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T07:05:12.779 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T07:05:12.819 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T07:05:12.963 INFO:teuthology.orchestra.run.smithi005.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4775-g7f24e589-1jammy [130 kB] 2024-09-14T07:05:12.966 INFO:teuthology.orchestra.run.smithi005.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4775-g7f24e589-1jammy [39.0 kB] 2024-09-14T07:05:12.967 INFO:teuthology.orchestra.run.smithi005.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4775-g7f24e589-1jammy [15.9 MB] 2024-09-14T07:05:13.012 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua5.1. 2024-09-14T07:05:13.038 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T07:05:13.078 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T07:05:13.156 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua5.1. 2024-09-14T07:05:13.182 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T07:05:13.222 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T07:05:13.344 INFO:teuthology.orchestra.run.smithi005.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4775-g7f24e589-1jammy [91.3 kB] 2024-09-14T07:05:13.345 INFO:teuthology.orchestra.run.smithi005.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.3.0-4775-g7f24e589-1jammy [178 kB] 2024-09-14T07:05:13.390 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua-any. 2024-09-14T07:05:13.405 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-14T07:05:13.440 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T07:05:13.517 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua-any. 2024-09-14T07:05:13.543 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-14T07:05:13.583 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T07:05:13.684 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package zip. 2024-09-14T07:05:13.699 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-14T07:05:13.742 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T07:05:13.844 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package zip. 2024-09-14T07:05:13.860 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-14T07:05:13.903 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T07:05:13.950 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 251 MB in 18s (14.2 MB/s) 2024-09-14T07:05:14.104 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package unzip. 2024-09-14T07:05:14.130 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T07:05:14.170 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-14T07:05:14.170 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T07:05:14.223 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package unzip. 2024-09-14T07:05:14.238 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T07:05:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-14T07:05:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-14T07:05:14.281 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T07:05:14.377 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T07:05:14.524 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package luarocks. 2024-09-14T07:05:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T07:05:14.574 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T07:05:14.644 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package luarocks. 2024-09-14T07:05:14.669 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T07:05:14.671 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-14T07:05:14.698 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-14T07:05:14.701 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T07:05:14.855 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T07:05:14.949 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package librgw2. 2024-09-14T07:05:14.975 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../026-librgw2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:15.015 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:15.084 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package librgw2. 2024-09-14T07:05:15.111 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../026-librgw2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:15.150 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:15.183 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-14T07:05:15.209 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-14T07:05:15.249 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T07:05:15.634 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T07:05:15.649 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:15.686 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-14T07:05:15.692 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:15.713 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:15.761 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:15.770 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T07:05:15.785 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:15.828 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:16.029 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T07:05:16.055 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T07:05:16.095 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T07:05:16.156 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-14T07:05:16.164 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T07:05:16.182 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:16.190 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T07:05:16.222 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:16.231 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T07:05:16.382 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T07:05:16.408 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:16.440 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:16.492 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T07:05:16.518 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:16.550 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:16.567 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-14T07:05:16.594 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-14T07:05:16.634 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:05:16.835 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-common. 2024-09-14T07:05:16.862 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:16.902 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:16.953 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-14T07:05:16.979 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-common. 2024-09-14T07:05:16.980 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-14T07:05:17.005 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:17.020 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T07:05:17.045 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:17.356 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librados2. 2024-09-14T07:05:17.383 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../007-librados2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:17.422 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:17.850 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libnbd0. 2024-09-14T07:05:17.877 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-14T07:05:17.909 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-14T07:05:18.220 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librbd1. 2024-09-14T07:05:18.247 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../009-librbd1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:18.316 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:18.441 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-base. 2024-09-14T07:05:18.468 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:18.517 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:18.559 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-base. 2024-09-14T07:05:18.585 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:18.643 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:18.914 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libcephfs2. 2024-09-14T07:05:18.941 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:18.981 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:19.095 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T07:05:19.121 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T07:05:19.161 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T07:05:19.205 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T07:05:19.231 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T07:05:19.250 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rados. 2024-09-14T07:05:19.266 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:19.271 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T07:05:19.300 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:19.423 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T07:05:19.450 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T07:05:19.489 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T07:05:19.574 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T07:05:19.601 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T07:05:19.640 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T07:05:19.669 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-14T07:05:19.696 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:19.744 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:19.801 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T07:05:19.827 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T07:05:19.867 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T07:05:19.969 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T07:05:19.995 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T07:05:20.035 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T07:05:20.038 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cephfs. 2024-09-14T07:05:20.066 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:20.105 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:20.170 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T07:05:20.197 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T07:05:20.237 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T07:05:20.313 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T07:05:20.341 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T07:05:20.380 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T07:05:20.424 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-14T07:05:20.452 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:20.499 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:20.548 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T07:05:20.574 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T07:05:20.615 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T07:05:20.649 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T07:05:20.676 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T07:05:20.716 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T07:05:20.761 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-14T07:05:20.788 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-14T07:05:20.819 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T07:05:20.892 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T07:05:20.908 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T07:05:20.942 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T07:05:21.019 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T07:05:21.045 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T07:05:21.085 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T07:05:21.130 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-prettytable. 2024-09-14T07:05:21.157 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-14T07:05:21.196 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-14T07:05:21.212 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-portend. 2024-09-14T07:05:21.238 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-14T07:05:21.278 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T07:05:21.388 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-portend. 2024-09-14T07:05:21.414 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-14T07:05:21.455 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T07:05:21.482 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rbd. 2024-09-14T07:05:21.498 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:21.549 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:21.582 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T07:05:21.608 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T07:05:21.671 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T07:05:21.758 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T07:05:21.784 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T07:05:21.824 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T07:05:21.860 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-14T07:05:21.887 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-14T07:05:21.918 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T07:05:21.993 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T07:05:22.019 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T07:05:22.059 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T07:05:22.111 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T07:05:22.137 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T07:05:22.154 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-14T07:05:22.169 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T07:05:22.170 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-14T07:05:22.213 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T07:05:22.413 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T07:05:22.439 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T07:05:22.479 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T07:05:22.490 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-14T07:05:22.514 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T07:05:22.518 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-14T07:05:22.541 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T07:05:22.557 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T07:05:22.581 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T07:05:22.791 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T07:05:22.817 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T07:05:22.857 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T07:05:22.877 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua5.1. 2024-09-14T07:05:22.904 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-14T07:05:22.943 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-14T07:05:23.092 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T07:05:23.119 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T07:05:23.135 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T07:05:23.158 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T07:05:23.162 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T07:05:23.194 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T07:05:23.238 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua-any. 2024-09-14T07:05:23.254 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-14T07:05:23.288 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-14T07:05:23.470 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T07:05:23.471 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-mako. 2024-09-14T07:05:23.496 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T07:05:23.498 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T07:05:23.532 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package zip. 2024-09-14T07:05:23.536 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T07:05:23.538 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T07:05:23.548 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-14T07:05:23.590 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking zip (3.0-12build2) ... 2024-09-14T07:05:23.831 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-mako. 2024-09-14T07:05:23.857 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T07:05:23.866 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T07:05:23.889 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T07:05:23.893 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T07:05:23.902 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package unzip. 2024-09-14T07:05:23.929 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-14T07:05:23.933 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T07:05:23.968 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-14T07:05:24.184 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T07:05:24.210 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T07:05:24.236 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T07:05:24.250 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T07:05:24.262 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T07:05:24.302 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T07:05:24.347 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package luarocks. 2024-09-14T07:05:24.374 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-14T07:05:24.414 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-14T07:05:24.553 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T07:05:24.580 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T07:05:24.580 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-webob. 2024-09-14T07:05:24.607 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T07:05:24.620 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T07:05:24.647 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T07:05:24.788 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librgw2. 2024-09-14T07:05:24.815 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../026-librgw2_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:24.854 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:24.923 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-webob. 2024-09-14T07:05:24.933 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T07:05:24.938 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T07:05:24.956 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T07:05:24.981 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T07:05:24.995 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T07:05:25.286 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T07:05:25.312 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T07:05:25.317 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T07:05:25.344 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T07:05:25.352 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T07:05:25.387 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T07:05:25.448 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rgw. 2024-09-14T07:05:25.464 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:25.498 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:25.672 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-paste. 2024-09-14T07:05:25.695 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T07:05:25.699 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T07:05:25.722 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T07:05:25.739 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T07:05:25.762 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T07:05:25.826 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-14T07:05:25.854 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-14T07:05:25.893 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T07:05:26.059 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T07:05:26.065 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-paste. 2024-09-14T07:05:26.075 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T07:05:26.091 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T07:05:26.117 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T07:05:26.131 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T07:05:26.204 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libradosstriper1. 2024-09-14T07:05:26.231 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:26.271 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:26.403 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T07:05:26.430 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T07:05:26.434 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T07:05:26.461 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T07:05:26.470 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T07:05:26.501 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T07:05:26.632 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-common. 2024-09-14T07:05:26.659 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:26.690 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:26.754 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T07:05:26.769 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T07:05:26.773 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T07:05:26.799 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T07:05:26.804 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T07:05:26.848 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T07:05:27.107 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T07:05:27.134 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T07:05:27.142 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T07:05:27.169 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T07:05:27.173 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T07:05:27.200 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T07:05:27.484 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T07:05:27.511 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T07:05:27.551 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T07:05:27.580 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T07:05:27.607 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T07:05:27.647 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T07:05:27.923 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T07:05:27.950 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T07:05:27.958 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-idna. 2024-09-14T07:05:27.985 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T07:05:27.989 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T07:05:28.025 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T07:05:28.287 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-base. 2024-09-14T07:05:28.309 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-idna. 2024-09-14T07:05:28.314 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:28.336 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T07:05:28.363 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:28.375 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T07:05:28.536 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T07:05:28.563 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T07:05:28.661 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T07:05:28.688 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T07:05:28.969 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T07:05:28.978 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T07:05:29.690 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-requests. 2024-09-14T07:05:29.716 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T07:05:29.748 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T07:05:29.867 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-14T07:05:29.873 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-requests. 2024-09-14T07:05:29.894 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-14T07:05:29.900 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T07:05:29.933 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-14T07:05:29.939 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T07:05:30.002 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T07:05:30.028 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T07:05:30.093 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T07:05:30.228 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cheroot. 2024-09-14T07:05:30.251 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T07:05:30.255 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-14T07:05:30.278 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T07:05:30.294 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T07:05:30.317 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T07:05:30.371 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T07:05:30.386 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:30.420 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:30.580 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-14T07:05:30.596 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-14T07:05:30.604 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T07:05:30.630 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-14T07:05:30.631 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:30.670 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:30.801 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T07:05:30.827 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:30.867 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:30.908 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-14T07:05:30.935 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-14T07:05:30.983 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-14T07:05:30.984 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T07:05:31.010 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:31.050 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T07:05:31.180 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T07:05:31.220 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T07:05:31.286 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-14T07:05:31.313 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-14T07:05:31.353 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-14T07:05:31.370 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T07:05:31.397 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T07:05:31.436 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T07:05:31.464 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T07:05:31.480 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T07:05:31.514 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T07:05:31.655 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-tempora. 2024-09-14T07:05:31.682 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-14T07:05:31.722 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-14T07:05:31.739 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T07:05:31.766 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T07:05:31.806 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T07:05:31.851 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T07:05:31.877 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:31.917 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:31.983 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-portend. 2024-09-14T07:05:32.010 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-14T07:05:32.050 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-14T07:05:32.134 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T07:05:32.160 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:32.192 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:32.304 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T07:05:32.330 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:32.370 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:32.377 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-14T07:05:32.404 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-14T07:05:32.444 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-14T07:05:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T07:05:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:32.747 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-14T07:05:32.774 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-14T07:05:32.813 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-14T07:05:32.924 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T07:05:32.951 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:32.982 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:33.127 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-natsort. 2024-09-14T07:05:33.154 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-14T07:05:33.193 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-14T07:05:33.265 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T07:05:33.292 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:33.332 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:33.505 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-logutils. 2024-09-14T07:05:33.532 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-14T07:05:33.571 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-14T07:05:33.874 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-14T07:05:33.901 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-14T07:05:33.941 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-14T07:05:34.052 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph. 2024-09-14T07:05:34.067 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../067-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:34.110 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:34.177 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-mako. 2024-09-14T07:05:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-14T07:05:34.235 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T07:05:34.410 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph. 2024-09-14T07:05:34.429 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T07:05:34.437 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../067-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:34.456 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:34.468 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:34.504 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:34.538 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-14T07:05:34.565 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-14T07:05:34.605 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-14T07:05:34.796 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T07:05:34.823 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:34.833 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T07:05:34.860 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:34.862 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:34.891 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:34.891 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-14T07:05:34.918 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-14T07:05:34.957 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-14T07:05:35.240 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T07:05:35.260 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-webob. 2024-09-14T07:05:35.267 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:35.277 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package cephadm. 2024-09-14T07:05:35.288 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-14T07:05:35.303 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../070-cephadm_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:35.306 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:35.335 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T07:05:35.343 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:35.646 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T07:05:35.671 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-waitress. 2024-09-14T07:05:35.673 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T07:05:35.699 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-14T07:05:35.713 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T07:05:35.718 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package cephadm. 2024-09-14T07:05:35.745 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../070-cephadm_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:35.766 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T07:05:35.784 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:36.046 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T07:05:36.049 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-tempita. 2024-09-14T07:05:36.057 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T07:05:36.061 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T07:05:36.074 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T07:05:36.076 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-14T07:05:36.095 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T07:05:36.107 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T07:05:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T07:05:36.377 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-paste. 2024-09-14T07:05:36.404 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-14T07:05:36.423 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T07:05:36.450 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T07:05:36.468 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T07:05:36.490 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T07:05:36.780 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-14T07:05:36.807 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-14T07:05:36.846 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T07:05:37.091 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-14T07:05:37.118 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-14T07:05:37.121 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-babel. 2024-09-14T07:05:37.148 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T07:05:37.157 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-14T07:05:37.187 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T07:05:37.462 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-babel. 2024-09-14T07:05:37.468 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-webtest. 2024-09-14T07:05:37.489 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T07:05:37.496 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-14T07:05:37.507 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T07:05:37.528 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T07:05:37.536 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T07:05:37.543 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-14T07:05:37.573 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T07:05:37.813 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pecan. 2024-09-14T07:05:37.814 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T07:05:37.840 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-14T07:05:37.841 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T07:05:37.872 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T07:05:37.879 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T07:05:37.918 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T07:05:37.945 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:37.985 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:38.201 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T07:05:38.201 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-certifi. 2024-09-14T07:05:38.217 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-14T07:05:38.227 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:38.251 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-14T07:05:38.267 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:38.322 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T07:05:38.348 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T07:05:38.388 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T07:05:38.537 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-idna. 2024-09-14T07:05:38.564 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-14T07:05:38.603 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T07:05:38.691 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-routes. 2024-09-14T07:05:38.718 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T07:05:38.757 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T07:05:38.839 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T07:05:38.866 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T07:05:38.906 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T07:05:38.915 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-urllib3. 2024-09-14T07:05:38.942 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-14T07:05:38.981 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T07:05:39.208 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-routes. 2024-09-14T07:05:39.235 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T07:05:39.275 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T07:05:39.306 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-requests. 2024-09-14T07:05:39.311 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T07:05:39.334 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-14T07:05:39.338 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:39.367 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T07:05:39.377 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:39.628 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T07:05:39.645 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-14T07:05:39.655 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:39.673 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-14T07:05:39.694 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T07:05:40.007 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-14T07:05:40.034 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:40.073 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:40.420 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-14T07:05:40.447 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:40.478 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:40.772 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-14T07:05:40.800 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-14T07:05:40.839 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-14T07:05:41.142 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-openssl. 2024-09-14T07:05:41.169 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-14T07:05:41.208 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-14T07:05:41.553 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr. 2024-09-14T07:05:41.580 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:41.620 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:41.973 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mon. 2024-09-14T07:05:42.000 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:42.031 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:42.116 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T07:05:42.133 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T07:05:42.174 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T07:05:42.499 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T07:05:42.517 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T07:05:42.558 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T07:05:42.678 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T07:05:42.705 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T07:05:42.726 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-osd. 2024-09-14T07:05:42.744 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T07:05:42.753 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:42.792 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:43.053 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T07:05:43.081 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T07:05:43.089 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T07:05:43.117 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T07:05:43.119 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T07:05:43.155 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T07:05:43.475 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T07:05:43.498 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T07:05:43.502 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T07:05:43.526 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T07:05:43.589 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T07:05:43.635 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T07:05:43.870 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph. 2024-09-14T07:05:43.886 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../067-ceph_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:43.900 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T07:05:43.928 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T07:05:43.929 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:44.052 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T07:05:44.239 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-fuse. 2024-09-14T07:05:44.267 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:44.306 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:44.353 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T07:05:44.381 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:44.420 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:44.609 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mds. 2024-09-14T07:05:44.637 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:44.720 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T07:05:44.748 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:44.778 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:44.803 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:45.229 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package cephadm. 2024-09-14T07:05:45.256 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../070-cephadm_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:45.295 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:45.473 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T07:05:45.501 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T07:05:45.548 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T07:05:45.606 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-14T07:05:45.633 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-14T07:05:45.672 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-14T07:05:45.731 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T07:05:45.759 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T07:05:45.798 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T07:05:45.885 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T07:05:45.912 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T07:05:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-14T07:05:45.951 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T07:05:45.966 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T07:05:46.025 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T07:05:46.084 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T07:05:46.112 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T07:05:46.150 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T07:05:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T07:05:46.307 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T07:05:46.346 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T07:05:46.496 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T07:05:46.523 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T07:05:46.562 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T07:05:46.657 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T07:05:46.685 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-14T07:05:46.732 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T07:05:46.857 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T07:05:46.884 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-14T07:05:46.923 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T07:05:47.047 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-babel. 2024-09-14T07:05:47.063 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-14T07:05:47.086 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T07:05:47.105 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T07:05:47.114 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T07:05:47.161 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T07:05:47.252 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T07:05:47.280 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T07:05:47.318 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T07:05:47.391 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jinja2. 2024-09-14T07:05:47.418 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-14T07:05:47.458 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T07:05:47.548 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T07:05:47.575 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T07:05:47.605 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T07:05:47.622 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T07:05:47.633 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T07:05:47.663 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T07:05:47.769 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-14T07:05:47.796 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:47.835 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:47.950 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T07:05:47.967 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T07:05:47.978 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T07:05:47.995 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T07:05:48.016 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T07:05:48.034 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T07:05:48.181 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-14T07:05:48.208 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-14T07:05:48.247 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-14T07:05:48.311 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T07:05:48.339 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T07:05:48.362 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T07:05:48.377 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T07:05:48.390 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T07:05:48.429 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T07:05:48.516 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-routes. 2024-09-14T07:05:48.532 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-14T07:05:48.566 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T07:05:48.877 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-14T07:05:48.905 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:48.944 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:49.066 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T07:05:49.083 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:49.101 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T07:05:49.125 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:49.137 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:49.168 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:49.436 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T07:05:49.464 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T07:05:49.502 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T07:05:49.654 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T07:05:49.682 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T07:05:49.721 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T07:05:49.789 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T07:05:49.817 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T07:05:49.855 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T07:05:50.207 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T07:05:50.225 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package jq. 2024-09-14T07:05:50.235 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T07:05:50.253 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T07:05:50.273 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T07:05:50.291 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T07:05:50.544 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package socat. 2024-09-14T07:05:50.551 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package jq. 2024-09-14T07:05:50.567 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T07:05:50.572 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T07:05:50.609 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T07:05:50.610 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T07:05:50.846 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package socat. 2024-09-14T07:05:50.861 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T07:05:50.895 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T07:05:50.922 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T07:05:50.938 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T07:05:50.972 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T07:05:51.157 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T07:05:51.173 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T07:05:51.207 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T07:05:51.343 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-test. 2024-09-14T07:05:51.359 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:51.401 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:51.546 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-test. 2024-09-14T07:05:51.574 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:51.613 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:51.849 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-14T07:05:51.867 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-14T07:05:51.907 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T07:05:52.352 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-joblib. 2024-09-14T07:05:52.380 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-14T07:05:52.410 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T07:05:52.797 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-14T07:05:52.825 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-14T07:05:52.863 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-14T07:05:53.166 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-sklearn. 2024-09-14T07:05:53.194 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-14T07:05:53.318 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T07:05:53.897 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T07:05:53.925 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:53.955 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:54.044 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-14T07:05:54.073 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:54.110 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:54.250 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T07:05:54.278 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:54.316 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:54.575 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T07:05:54.587 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T07:05:54.603 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:54.604 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:05:54.637 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:54.642 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:54.931 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T07:05:54.945 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T07:05:54.960 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T07:05:54.972 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T07:05:54.997 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T07:05:55.011 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T07:05:55.030 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cachetools. 2024-09-14T07:05:55.059 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-14T07:05:55.088 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-14T07:05:55.345 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T07:05:55.352 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T07:05:55.373 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T07:05:55.380 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T07:05:55.407 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T07:05:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T07:05:55.600 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-14T07:05:55.628 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-14T07:05:55.666 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-14T07:05:55.786 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T07:05:55.798 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T07:05:55.814 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T07:05:55.825 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T07:05:55.853 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T07:05:55.864 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T07:05:55.978 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-14T07:05:56.006 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-14T07:05:56.044 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T07:05:56.258 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package pkg-config. 2024-09-14T07:05:56.287 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T07:05:56.319 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package pkg-config. 2024-09-14T07:05:56.324 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T07:05:56.347 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T07:05:56.355 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rsa. 2024-09-14T07:05:56.384 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-14T07:05:56.385 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T07:05:56.422 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-14T07:05:56.586 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T07:05:56.613 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T07:05:56.614 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T07:05:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T07:05:56.652 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T07:05:56.671 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T07:05:56.709 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-google-auth. 2024-09-14T07:05:56.726 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-14T07:05:56.767 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-14T07:05:57.046 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-14T07:05:57.074 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-14T07:05:57.080 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T07:05:57.091 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T07:05:57.103 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T07:05:57.108 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T07:05:57.119 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T07:05:57.147 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T07:05:57.157 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T07:05:57.400 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T07:05:57.427 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T07:05:57.440 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-websocket. 2024-09-14T07:05:57.457 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T07:05:57.468 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-14T07:05:57.469 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T07:05:57.497 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T07:05:57.506 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-14T07:05:57.535 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T07:05:57.719 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T07:05:57.735 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T07:05:57.777 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T07:05:57.826 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-14T07:05:57.855 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-14T07:05:57.855 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T07:05:57.883 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T07:05:57.892 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T07:05:57.922 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T07:05:58.122 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T07:05:58.151 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T07:05:58.189 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T07:05:58.233 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T07:05:58.261 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T07:05:58.299 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T07:05:58.508 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-py. 2024-09-14T07:05:58.536 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-14T07:05:58.573 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-14T07:05:58.574 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T07:05:58.590 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:05:58.633 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:05:58.652 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-py. 2024-09-14T07:05:58.680 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-14T07:05:58.719 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T07:05:58.903 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T07:05:58.931 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T07:05:58.951 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-14T07:05:58.969 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T07:05:58.979 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-14T07:05:59.017 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T07:05:59.047 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T07:05:59.075 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T07:05:59.113 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T07:05:59.337 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-14T07:05:59.366 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T07:05:59.404 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T07:05:59.432 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T07:05:59.460 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T07:05:59.498 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T07:05:59.568 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T07:05:59.596 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T07:05:59.634 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T07:05:59.673 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package jq. 2024-09-14T07:05:59.690 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-14T07:05:59.731 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-14T07:05:59.793 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-toml. 2024-09-14T07:05:59.821 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-14T07:05:59.860 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T07:05:59.953 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-toml. 2024-09-14T07:05:59.981 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-14T07:05:59.984 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package socat. 2024-09-14T07:06:00.001 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-14T07:06:00.020 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T07:06:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-14T07:06:00.104 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T07:06:00.132 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T07:06:00.170 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T07:06:00.345 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package xmlstarlet. 2024-09-14T07:06:00.347 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T07:06:00.362 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-14T07:06:00.376 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T07:06:00.412 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-14T07:06:00.414 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T07:06:00.516 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T07:06:00.544 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T07:06:00.582 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T07:06:00.742 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T07:06:00.770 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T07:06:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T07:06:00.851 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T07:06:00.867 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T07:06:00.901 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T07:06:00.999 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-test. 2024-09-14T07:06:01.016 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:01.057 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:01.061 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T07:06:01.077 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T07:06:01.120 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T07:06:01.622 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package radosgw. 2024-09-14T07:06:01.650 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../117-radosgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:01.688 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:01.790 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package radosgw. 2024-09-14T07:06:01.818 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../117-radosgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:01.856 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:02.592 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T07:06:02.620 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:02.658 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:02.809 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T07:06:02.825 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:02.867 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:02.911 INFO:teuthology.orchestra.run.smithi145.stdout:Selecting previously unselected package rbd-nbd. 2024-09-14T07:06:02.939 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing to unpack .../119-rbd-nbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:02.977 INFO:teuthology.orchestra.run.smithi145.stdout:Unpacking rbd-nbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:03.120 INFO:teuthology.orchestra.run.smithi110.stdout:Selecting previously unselected package rbd-nbd. 2024-09-14T07:06:03.136 INFO:teuthology.orchestra.run.smithi110.stdout:Preparing to unpack .../119-rbd-nbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:03.178 INFO:teuthology.orchestra.run.smithi110.stdout:Unpacking rbd-nbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:03.290 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T07:06:03.491 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T07:06:03.540 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T07:06:03.570 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-volume. 2024-09-14T07:06:03.599 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4775-g7f24e589-1jammy_all.deb ... 2024-09-14T07:06:03.645 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:03.650 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T07:06:03.733 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T07:06:03.859 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T07:06:03.891 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T07:06:03.957 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-14T07:06:03.986 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:04.023 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:04.153 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T07:06:04.221 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T07:06:04.301 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-14T07:06:04.318 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-14T07:06:04.360 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T07:06:04.470 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T07:06:04.647 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-14T07:06:04.659 INFO:teuthology.orchestra.run.smithi145.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T07:06:04.676 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-14T07:06:04.705 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-14T07:06:04.726 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:04.927 INFO:teuthology.orchestra.run.smithi110.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T07:06:05.028 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:05.067 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package nvme-cli. 2024-09-14T07:06:05.096 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-14T07:06:05.142 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T07:06:05.234 INFO:teuthology.orchestra.run.smithi145.stdout:Adding system user cephadm....done 2024-09-14T07:06:05.384 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T07:06:05.556 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package pkg-config. 2024-09-14T07:06:05.560 INFO:teuthology.orchestra.run.smithi110.stdout:Adding system user cephadm....done 2024-09-14T07:06:05.585 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-14T07:06:05.622 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T07:06:05.642 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T07:06:05.717 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T07:06:05.875 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-14T07:06:05.892 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-14T07:06:05.893 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T07:06:05.933 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-14T07:06:05.968 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T07:06:06.019 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T07:06:06.236 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T07:06:06.279 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T07:06:06.345 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-14T07:06:06.362 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T07:06:06.374 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-14T07:06:06.411 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-14T07:06:06.530 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T07:06:06.597 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T07:06:06.656 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T07:06:06.664 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pastescript. 2024-09-14T07:06:06.693 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-14T07:06:06.722 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-14T07:06:06.839 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T07:06:06.965 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T07:06:06.966 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T07:06:07.017 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pluggy. 2024-09-14T07:06:07.046 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-14T07:06:07.083 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-14T07:06:07.267 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T07:06:07.309 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T07:06:07.403 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-psutil. 2024-09-14T07:06:07.432 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-14T07:06:07.469 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-14T07:06:07.553 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T07:06:07.611 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T07:06:07.789 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-py. 2024-09-14T07:06:07.818 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-14T07:06:07.829 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T07:06:07.855 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-14T07:06:07.862 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T07:06:08.081 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T07:06:08.121 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T07:06:08.201 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pygments. 2024-09-14T07:06:08.230 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-14T07:06:08.267 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T07:06:08.340 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T07:06:08.381 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T07:06:08.599 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T07:06:08.640 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T07:06:08.713 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-14T07:06:08.742 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-14T07:06:08.779 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-14T07:06:08.875 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:08.917 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T07:06:09.090 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-toml. 2024-09-14T07:06:09.119 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-14T07:06:09.135 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T07:06:09.157 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-14T07:06:09.159 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:09.344 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-14T07:06:09.419 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T07:06:09.460 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pytest. 2024-09-14T07:06:09.470 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T07:06:09.488 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-14T07:06:09.520 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-14T07:06:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T07:06:09.605 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T07:06:09.646 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T07:06:09.772 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T07:06:09.880 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-simplejson. 2024-09-14T07:06:09.908 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-14T07:06:09.914 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T07:06:09.946 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-14T07:06:10.055 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T07:06:10.207 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-14T07:06:10.224 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-14T07:06:10.248 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T07:06:10.265 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-14T07:06:10.315 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T07:06:10.620 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T07:06:10.654 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T07:06:10.706 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T07:06:10.710 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T07:06:10.737 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T07:06:10.801 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T07:06:10.836 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T07:06:10.902 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T07:06:11.011 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package radosgw. 2024-09-14T07:06:11.041 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../117-radosgw_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:11.204 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T07:06:11.245 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T07:06:11.252 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:11.305 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T07:06:11.371 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T07:06:11.556 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T07:06:11.664 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T07:06:11.840 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T07:06:11.932 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T07:06:12.133 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T07:06:12.247 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package rbd-fuse. 2024-09-14T07:06:12.259 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T07:06:12.264 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:12.266 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T07:06:12.305 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:12.392 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T07:06:12.566 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package rbd-nbd. 2024-09-14T07:06:12.595 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../119-rbd-nbd_19.3.0-4775-g7f24e589-1jammy_amd64.deb ... 2024-09-14T07:06:12.632 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking rbd-nbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:12.930 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-14T07:06:12.978 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T07:06:13.027 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T07:06:13.164 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-14T07:06:13.237 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T07:06:13.290 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-14T07:06:13.312 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T07:06:13.363 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T07:06:13.438 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T07:06:13.489 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T07:06:13.564 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T07:06:13.565 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-14T07:06:13.781 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T07:06:13.873 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T07:06:13.894 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-14T07:06:14.174 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T07:06:14.258 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T07:06:14.314 INFO:teuthology.orchestra.run.smithi005.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-14T07:06:14.384 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:14.417 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T07:06:14.500 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T07:06:14.802 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T07:06:14.858 INFO:teuthology.orchestra.run.smithi005.stdout:Adding system user cephadm....done 2024-09-14T07:06:14.869 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T07:06:15.006 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-14T07:06:15.095 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T07:06:15.137 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T07:06:15.424 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-14T07:06:15.479 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T07:06:15.497 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T07:06:15.684 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-14T07:06:15.731 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T07:06:15.764 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T07:06:15.801 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-14T07:06:15.848 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:15.924 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:16.044 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-14T07:06:16.149 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T07:06:16.225 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T07:06:16.362 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-14T07:06:16.467 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T07:06:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-14T07:06:16.518 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T07:06:16.798 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-14T07:06:17.175 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-14T07:06:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-14T07:06:17.565 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T07:06:17.686 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-14T07:06:17.707 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T07:06:17.799 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:17.913 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-14T07:06:17.917 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T07:06:17.933 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:18.068 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T07:06:18.160 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T07:06:18.181 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-14T07:06:18.286 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T07:06:18.319 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T07:06:18.412 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T07:06:18.437 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T07:06:18.448 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-14T07:06:18.563 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T07:06:18.705 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T07:06:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-ceph-argparse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:18.864 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T07:06:18.956 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T07:06:19.050 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-14T07:06:19.124 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T07:06:19.160 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-14T07:06:19.207 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:19.269 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-14T07:06:19.358 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T07:06:19.378 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-14T07:06:19.383 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:19.504 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-14T07:06:19.509 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T07:06:19.651 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T07:06:19.763 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-14T07:06:19.785 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T07:06:19.902 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T07:06:20.053 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T07:06:20.137 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-14T07:06:20.170 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-14T07:06:20.170 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T07:06:20.225 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-14T07:06:20.312 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T07:06:20.343 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-14T07:06:20.421 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T07:06:20.572 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T07:06:20.655 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T07:06:20.719 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-14T07:06:20.823 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T07:06:20.845 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-14T07:06:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T07:06:21.049 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T07:06:21.090 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T07:06:21.121 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-14T07:06:21.191 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T07:06:21.325 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T07:06:21.398 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-14T07:06:21.425 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T07:06:21.459 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T07:06:21.569 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T07:06:21.702 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T07:06:21.732 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up zip (3.0-12build2) ... 2024-09-14T07:06:21.858 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-14T07:06:21.862 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T07:06:22.003 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T07:06:22.171 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T07:06:22.262 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T07:06:22.363 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T07:06:22.447 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T07:06:22.464 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T07:06:22.535 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-14T07:06:22.545 INFO:teuthology.orchestra.run.smithi110.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T07:06:22.573 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T07:06:22.699 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T07:06:22.771 INFO:teuthology.orchestra.run.smithi145.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T07:06:22.782 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T07:06:22.811 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-14T07:06:22.884 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T07:06:22.937 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-14T07:06:23.042 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T07:06:23.063 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-14T07:06:23.143 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T07:06:23.260 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T07:06:23.364 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-14T07:06:23.520 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T07:06:23.545 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T07:06:23.645 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T07:06:23.757 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-14T07:06:23.813 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T07:06:23.888 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:23.939 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T07:06:24.000 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-14T07:06:24.005 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T07:06:24.198 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:24.307 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T07:06:24.324 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T07:06:24.368 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-14T07:06:24.608 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T07:06:24.650 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T07:06:24.653 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-14T07:06:24.875 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T07:06:24.926 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T07:06:24.984 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T07:06:25.021 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-14T07:06:25.186 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T07:06:25.253 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T07:06:25.272 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-14T07:06:25.314 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T07:06:25.398 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:25.590 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T07:06:25.597 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T07:06:25.733 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-14T07:06:25.857 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T07:06:25.968 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T07:06:26.068 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-14T07:06:26.226 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T07:06:26.261 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T07:06:26.510 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T07:06:26.654 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T07:06:26.846 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T07:06:26.939 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T07:06:26.980 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:27.090 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:27.216 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T07:06:27.275 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T07:06:27.307 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-14T07:06:27.385 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:27.511 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:27.525 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:27.551 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:27.634 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-14T07:06:27.637 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T07:06:27.686 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:27.812 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T07:06:27.902 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-14T07:06:27.973 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.020 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-14T07:06:28.080 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.099 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.146 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-14T07:06:28.206 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.209 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T07:06:28.332 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.456 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-14T07:06:28.485 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.534 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T07:06:28.534 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T07:06:28.611 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.724 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-14T07:06:28.737 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.952 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T07:06:28.952 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T07:06:28.975 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:28.983 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:29.076 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:29.084 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-14T07:06:29.202 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:29.336 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T07:06:29.385 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-14T07:06:29.489 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:29.615 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:29.653 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-14T07:06:29.741 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:29.850 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T07:06:29.913 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-14T07:06:30.181 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-14T07:06:30.440 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-14T07:06:30.482 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:30.599 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:30.700 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up rbd-nbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:30.716 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-14T07:06:30.818 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:30.842 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-14T07:06:30.935 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.012 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.070 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.118 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-14T07:06:31.146 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.219 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-14T07:06:31.273 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up rbd-nbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.398 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.512 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-14T07:06:31.524 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.604 INFO:teuthology.orchestra.run.smithi110.stdout:Adding group ceph....done 2024-09-14T07:06:31.642 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:31.821 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-14T07:06:31.939 INFO:teuthology.orchestra.run.smithi110.stdout:Adding system user ceph....done 2024-09-14T07:06:32.081 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-14T07:06:32.115 INFO:teuthology.orchestra.run.smithi110.stdout:Setting system user ceph properties....done 2024-09-14T07:06:32.134 INFO:teuthology.orchestra.run.smithi110.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T07:06:32.152 INFO:teuthology.orchestra.run.smithi145.stdout:Adding group ceph....done 2024-09-14T07:06:32.207 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-14T07:06:32.270 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T07:06:32.333 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-14T07:06:32.422 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-14T07:06:32.528 INFO:teuthology.orchestra.run.smithi145.stdout:Adding system user ceph....done 2024-09-14T07:06:32.556 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T07:06:32.660 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-14T07:06:32.688 INFO:teuthology.orchestra.run.smithi145.stdout:Setting system user ceph properties....done 2024-09-14T07:06:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T07:06:32.761 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-14T07:06:32.849 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T07:06:33.029 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:33.137 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-14T07:06:33.155 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:33.177 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T07:06:33.397 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-14T07:06:33.523 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-14T07:06:33.617 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T07:06:33.617 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T07:06:33.643 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:33.782 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-14T07:06:33.786 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:33.908 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-14T07:06:34.065 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:34.234 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-14T07:06:34.255 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T07:06:34.255 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T07:06:34.438 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T07:06:34.535 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-14T07:06:34.738 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:34.795 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-14T07:06:34.904 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-14T07:06:35.026 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:35.111 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T07:06:35.171 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-14T07:06:35.264 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T07:06:35.265 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T07:06:35.533 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-14T07:06:35.587 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:35.744 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:35.801 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-14T07:06:35.880 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T07:06:35.880 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T07:06:35.961 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T07:06:35.961 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T07:06:36.161 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-14T07:06:36.346 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:36.425 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:36.446 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-14T07:06:36.575 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T07:06:36.576 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T07:06:36.700 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T07:06:36.701 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T07:06:36.807 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-14T07:06:36.950 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librados2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.068 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.069 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.169 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-14T07:06:37.209 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.336 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.348 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T07:06:37.348 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T07:06:37.487 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.488 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libcephfs2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.623 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libradosstriper1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.707 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T07:06:37.707 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T07:06:37.765 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-14T07:06:37.830 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:37.956 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.042 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librbd1 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.107 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.151 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-modules-core (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.238 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.252 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.319 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T07:06:38.319 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T07:06:38.363 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.455 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T07:06:38.455 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-14T07:06:38.498 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.649 INFO:teuthology.orchestra.run.smithi110.stdout:Setting up ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.817 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.951 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:38.987 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libcephfs-dev (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:39.052 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:39.077 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T07:06:39.155 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rados (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:39.186 INFO:teuthology.orchestra.run.smithi145.stdout:Setting up ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:39.194 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T07:06:39.306 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librgw2 (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:39.330 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T07:06:39.432 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-14T07:06:39.547 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T07:06:39.657 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T07:06:39.801 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T07:06:40.447 INFO:teuthology.orchestra.run.smithi110.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T07:06:40.609 INFO:teuthology.orchestra.run.smithi145.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T07:06:40.720 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:40.846 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up rbd-fuse (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:40.972 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up rbd-nbd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:41.098 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:41.224 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cephfs (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:41.350 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-common (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:41.551 INFO:teuthology.orchestra.run.smithi110.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T07:06:41.557 DEBUG:teuthology.parallel:result is None 2024-09-14T07:06:41.562 INFO:teuthology.orchestra.run.smithi145.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T07:06:41.567 DEBUG:teuthology.parallel:result is None 2024-09-14T07:06:42.179 INFO:teuthology.orchestra.run.smithi005.stdout:Adding group ceph....done 2024-09-14T07:06:42.487 INFO:teuthology.orchestra.run.smithi005.stdout:Adding system user ceph....done 2024-09-14T07:06:42.630 INFO:teuthology.orchestra.run.smithi005.stdout:Setting system user ceph properties....done 2024-09-14T07:06:42.649 INFO:teuthology.orchestra.run.smithi005.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-14T07:06:42.804 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-14T07:06:43.143 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-14T07:06:43.627 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-test (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:43.728 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up radosgw (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:44.220 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T07:06:44.220 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-14T07:06:44.704 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-base (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:45.069 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-14T07:06:45.565 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mds (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:45.799 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T07:06:45.799 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-14T07:06:46.283 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:46.511 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T07:06:46.511 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-14T07:06:46.984 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-osd (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:47.238 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T07:06:47.238 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-14T07:06:47.704 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:47.935 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:48.195 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mon (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:48.458 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T07:06:48.458 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-14T07:06:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-cephadm (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:49.193 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-dashboard (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:49.336 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-volume (19.3.0-4775-g7f24e589-1jammy) ... 2024-09-14T07:06:49.698 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-14T07:06:49.806 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-14T07:06:49.967 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-14T07:06:50.659 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-14T07:06:51.327 INFO:teuthology.orchestra.run.smithi005.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-14T07:06:51.332 DEBUG:teuthology.parallel:result is None 2024-09-14T07:06:51.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T07:06:51.480 DEBUG:teuthology.orchestra.run.smithi005:> dpkg-query -W -f '${Version}' ceph 2024-09-14T07:06:51.500 INFO:teuthology.orchestra.run.smithi005.stdout:19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:06:51.500 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:06:51.500 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775-g7f24e589-1jammy is installed. 2024-09-14T07:06:51.502 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T07:06:51.643 DEBUG:teuthology.orchestra.run.smithi110:> dpkg-query -W -f '${Version}' ceph 2024-09-14T07:06:51.662 INFO:teuthology.orchestra.run.smithi110.stdout:19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:06:51.663 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:06:51.663 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775-g7f24e589-1jammy is installed. 2024-09-14T07:06:51.665 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T07:06:51.807 DEBUG:teuthology.orchestra.run.smithi145:> dpkg-query -W -f '${Version}' ceph 2024-09-14T07:06:51.827 INFO:teuthology.orchestra.run.smithi145.stdout:19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:06:51.827 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775-g7f24e589-1jammy 2024-09-14T07:06:51.827 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775-g7f24e589-1jammy is installed. 2024-09-14T07:06:51.829 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T07:06:51.830 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:51.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T07:06:51.845 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:06:51.845 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T07:06:51.860 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:06:51.860 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T07:06:51.883 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T07:06:51.883 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:51.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T07:06:51.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T07:06:51.955 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:06:51.955 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T07:06:51.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T07:06:52.023 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:06:52.023 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T07:06:52.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T07:06:52.094 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T07:06:52.094 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:52.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T07:06:52.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T07:06:52.163 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:06:52.163 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T07:06:52.176 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T07:06:52.231 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:06:52.231 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T07:06:52.244 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T07:06:52.298 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T07:06:52.298 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:52.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T07:06:52.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T07:06:52.371 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:06:52.371 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T07:06:52.384 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T07:06:52.439 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:06:52.439 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T07:06:52.452 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T07:06:52.506 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T07:06:52.587 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T07:06:52.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2024-09-14T07:06:52.590 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /var/log/ceph 2024-09-14T07:06:52.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /var/log/ceph 2024-09-14T07:06:52.605 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T07:06:52.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T07:06:52.649 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T07:06:52.651 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T07:06:52.663 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T07:06:52.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T07:06:52.705 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T07:06:52.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T07:06:52.729 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T07:06:52.730 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T07:06:52.730 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414', 'branch': 'main', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900414', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 5839, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'sleep_before_teardown': 0, 'subset': '99/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a', 'targets': {'smithi005.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBHbPSdA5gs4Neklbw15b6Jz0kJPGBLCUU3u8DFEsXeoizVurScii8GxNVVsJol9YXZfl9DN+F69uPSbobVGnTo=', 'smithi110.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpbnw5uEgZQzlNMHRPssbUxIFZEHxLuZQarXi/6EcTsNNB7SW7YIBOiotcMpNvecbxNk9piwnGEmL7LLfapEwA=', 'smithi145.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBIEoZkAZgAE4Znx+kpj+O1M3bwG6hqbxJzoPMfn63ERTK3xKIN8oJYCSpp41Ytqsv+Hc8arLujyCZTKBHRIB2o='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_20:16:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T07:06:52.730 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T07:06:52.765 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T07:06:52.767 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T07:06:52.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T07:06:52.812 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T07:06:52.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T07:06:52.827 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:52.827 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T07:06:52.873 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T07:06:52.874 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-09-14T07:06:52.924 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T07:06:52.924 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:52.924 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-09-14T07:06:52.924 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:52.924 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-14 07:01:08.655434327 +0000 2024-09-14T07:06:52.925 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-14 07:01:08.415436470 +0000 2024-09-14T07:06:52.925 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-14 07:01:08.415436470 +0000 2024-09-14T07:06:52.925 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-09-14T07:06:52.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T07:06:52.978 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-14T07:06:52.978 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-14T07:06:52.978 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000322134 s, 1.6 MB/s 2024-09-14T07:06:52.980 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T07:06:53.029 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-14 07:01:08.887432254 +0000 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-14 07:01:08.883432290 +0000 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-14 07:01:08.883432290 +0000 2024-09-14T07:06:53.076 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-09-14T07:06:53.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T07:06:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-14T07:06:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-14T07:06:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000302217 s, 1.7 MB/s 2024-09-14T07:06:53.132 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T07:06:53.182 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-14 07:01:09.635425573 +0000 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-14 07:01:09.379427860 +0000 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-14 07:01:09.379427860 +0000 2024-09-14T07:06:53.232 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-09-14T07:06:53.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T07:06:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-14T07:06:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-14T07:06:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000296675 s, 1.7 MB/s 2024-09-14T07:06:53.291 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T07:06:53.341 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-09-14T07:06:53.387 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T07:06:53.388 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:53.388 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-09-14T07:06:53.388 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:53.388 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-14 07:01:09.879423393 +0000 2024-09-14T07:06:53.388 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-14 07:01:09.875423429 +0000 2024-09-14T07:06:53.388 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-14 07:01:09.875423429 +0000 2024-09-14T07:06:53.388 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-09-14T07:06:53.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T07:06:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-14T07:06:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-14T07:06:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000300954 s, 1.7 MB/s 2024-09-14T07:06:53.444 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T07:06:53.493 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-14T07:06:53.494 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:06:53.494 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T07:06:53.499 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T07:06:53.499 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-14T07:06:53.549 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T07:06:53.549 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:53.549 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-09-14T07:06:53.549 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:53.549 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-14 07:01:06.221967014 +0000 2024-09-14T07:06:53.549 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-14 07:01:06.217967049 +0000 2024-09-14T07:06:53.549 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-14 07:01:06.217967049 +0000 2024-09-14T07:06:53.550 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-14T07:06:53.550 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T07:06:53.602 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-14T07:06:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-14T07:06:53.603 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000279666 s, 1.8 MB/s 2024-09-14T07:06:53.604 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T07:06:53.654 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-14T07:06:53.700 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T07:06:53.700 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:53.700 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-09-14T07:06:53.700 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:53.700 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-14 07:01:06.977960261 +0000 2024-09-14T07:06:53.700 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-14 07:01:06.717962583 +0000 2024-09-14T07:06:53.701 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-14 07:01:06.717962583 +0000 2024-09-14T07:06:53.701 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-14T07:06:53.701 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T07:06:53.754 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-14T07:06:53.754 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-14T07:06:53.754 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000256654 s, 2.0 MB/s 2024-09-14T07:06:53.756 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T07:06:53.805 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-14 07:01:07.489955687 +0000 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-14 07:01:07.197958296 +0000 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-14 07:01:07.197958296 +0000 2024-09-14T07:06:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-14T07:06:53.853 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T07:06:53.906 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-14T07:06:53.906 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-14T07:06:53.906 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000305884 s, 1.7 MB/s 2024-09-14T07:06:53.907 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T07:06:53.957 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-14T07:06:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T07:06:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T07:06:54.004 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-14T07:06:54.004 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T07:06:54.004 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-14 07:01:07.949951579 +0000 2024-09-14T07:06:54.004 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-14 07:01:07.701953795 +0000 2024-09-14T07:06:54.004 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-14 07:01:07.701953795 +0000 2024-09-14T07:06:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-09-14T07:06:54.005 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T07:06:54.062 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-14T07:06:54.062 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-14T07:06:54.062 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000298522 s, 1.7 MB/s 2024-09-14T07:06:54.063 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T07:06:54.113 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-14T07:06:54.113 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-14T07:06:54.113 INFO:tasks.ceph:Generating config... 2024-09-14T07:06:54.115 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T07:06:54.115 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T07:06:54.115 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T07:06:54.115 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T07:06:54.115 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T07:06:54.115 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T07:06:54.115 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T07:06:54.115 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T07:06:54.115 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T07:06:54.115 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-14T07:06:54.115 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T07:06:54.116 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T07:06:54.117 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T07:06:54.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T07:06:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T07:06:54.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T07:06:54.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T07:06:54.295 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.c', '[v2:172.21.15.5:3301,v1:172.21.15.5:6790]'), ('mon.b', '172.21.15.110')] 2024-09-14T07:06:54.295 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.110', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T07:06:54.296 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:54.296 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T07:06:54.348 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --addv c '[v2:172.21.15.5:3301,v1:172.21.15.5:6790]' --add b 172.21.15.110 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:06:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:06:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid 717e3012-acdf-42c1-a4e0-d4c94afd9f15 2024-09-14T07:06:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-09-14T07:06:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2024-09-14T07:06:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 717e3012-acdf-42c1-a4e0-d4c94afd9f15 2024-09-14T07:06:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2024-09-14T07:06:54.436923+0000 2024-09-14T07:06:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-14T07:06:54.436923+0000 2024-09-14T07:06:54.437 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 17 (quincy) 2024-09-14T07:06:54.437 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2024-09-14T07:06:54.437 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-14T07:06:54.437 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-14T07:06:54.437 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.5:3301/0,v1:172.21.15.5:6790/0] mon.c 2024-09-14T07:06:54.437 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T07:06:54.438 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T07:06:54.443 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 717e3012-acdf-42c1-a4e0-d4c94afd9f15... 2024-09-14T07:06:54.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T07:06:54.488 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T07:06:54.491 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T07:06:54.514 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-09-14T07:06:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-09-14T07:06:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T07:06:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-09-14T07:06:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-09-14T07:06:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-09-14T07:06:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.110" 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon client directed command retry = 5 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on pool no app = False 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: ms inject socket failures = 5000 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 717e3012-acdf-42c1-a4e0-d4c94afd9f15 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-09-14T07:06:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-09-14T07:06:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore compression algorithm = zlib 2024-09-14T07:06:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore compression mode = aggressive 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 1/20 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 1/20 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 4/10 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd blocked scrub grace period = 3600 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd max markdown count = 1000 2024-09-14T07:06:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-09-14T07:06:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T07:06:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T07:06:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T07:06:54.525 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-09-14T07:06:54.525 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-09-14T07:06:54.525 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-09-14T07:06:54.525 INFO:teuthology.orchestra.run.smithi145.stdout:[global] 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: chdir = "" 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: auth supported = cephx 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: filestore xattr use omap = true 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: mon clock drift allowed = 1.000 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: osd crush chooseleaf type = 0 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: auth debug = true 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on old message = true 2024-09-14T07:06:54.526 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on bug = true 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: mon pg warn max object skew = 0 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default size = 2 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow primary affinity = true 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow pg remap = true 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on legacy crush tunables = false 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on crush straw calc version zero = false 2024-09-14T07:06:54.527 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on no sortbitwise = false 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on osd down out interval zero = false 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on too few osds = false 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: mon_allow_pool_size_one = true 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: osd default data pool replay window = 5 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: mon allow pool delete = true 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.528 INFO:teuthology.orchestra.run.smithi145.stdout: mon cluster log file level = debug 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: debug asserts on shutdown = true 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: mon health detail to clog = false 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: mon host = "172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.110" 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: mon client directed command retry = 5 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on pool no app = False 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: ms inject socket failures = 5000 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: fsid = 717e3012-acdf-42c1-a4e0-d4c94afd9f15 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout:[osd] 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: osd journal size = 100 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub load threshold = 5.0 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub max interval = 600 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: osd mclock profile = high_recovery_ops 2024-09-14T07:06:54.529 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd recover clone overlap = true 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd recovery max chunk = 1048576 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug shutdown = true 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug op order = true 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify stray on activate = true 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug trim objects = true 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd open classes on start = true 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug pg log writeout = true 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd deep scrub update digest min age = 30 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.530 INFO:teuthology.orchestra.run.smithi145.stdout: osd map max advance = 10 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: journal zero on create = true 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: filestore ondisk finisher threads = 3 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: filestore apply finisher threads = 3 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: bdev debug aio = true 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug misdirected ops = true 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore block size = 96636764160 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore compression algorithm = zlib 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore compression mode = aggressive 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore fsck on mount = True 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluefs = 1/20 2024-09-14T07:06:54.531 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluestore = 1/20 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: debug osd = 20 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: debug rocksdb = 4/10 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd full ratio = 0.9 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd nearfull ratio = 0.8 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: osd blocked scrub grace period = 3600 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: osd failsafe full ratio = 0.95 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: osd max markdown count = 1000 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: osd objectstore = bluestore 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr] 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-14T07:06:54.532 INFO:teuthology.orchestra.run.smithi145.stdout: debug mgr = 20 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: mgr/telemetry/nag = false 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout:[mon] 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: debug paxos = 20 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: mon data avail warn = 5 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: mon mgr mkfs grace = 240 2024-09-14T07:06:54.533 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd reporter subtree level = osd 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd prime pg temp = true 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: # 1m isn't quite enough 2024-09-14T07:06:54.534 INFO:teuthology.orchestra.run.smithi145.stdout: mon_down_mkfs_grace = 2m 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_filestore_osds = false 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout:[client] 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: rgw cache enabled = true 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable ops log = true 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable usage log = true 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.a] 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.c] 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.b] 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-09-14T07:06:54.535 INFO:teuthology.orchestra.run.smithi110.stdout: chdir = "" 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: auth supported = cephx 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: filestore xattr use omap = true 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: mon clock drift allowed = 1.000 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: osd crush chooseleaf type = 0 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: auth debug = true 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on old message = true 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on bug = true 2024-09-14T07:06:54.536 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: mon pg warn max object skew = 0 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default size = 2 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow primary affinity = true 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow pg remap = true 2024-09-14T07:06:54.537 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on legacy crush tunables = false 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on crush straw calc version zero = false 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on no sortbitwise = false 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on osd down out interval zero = false 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on too few osds = false 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: mon_allow_pool_size_one = true 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: osd default data pool replay window = 5 2024-09-14T07:06:54.538 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: mon allow pool delete = true 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: mon cluster log file level = debug 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: debug asserts on shutdown = true 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: mon health detail to clog = false 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: mon host = "172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.110" 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: mon client directed command retry = 5 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on pool no app = False 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: ms inject socket failures = 5000 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = 717e3012-acdf-42c1-a4e0-d4c94afd9f15 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout:[osd] 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: osd journal size = 100 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.539 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub load threshold = 5.0 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub max interval = 600 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd mclock profile = high_recovery_ops 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd recover clone overlap = true 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd recovery max chunk = 1048576 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug shutdown = true 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug op order = true 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify stray on activate = true 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug trim objects = true 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd open classes on start = true 2024-09-14T07:06:54.540 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug pg log writeout = true 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: osd deep scrub update digest min age = 30 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: osd map max advance = 10 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: journal zero on create = true 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: filestore ondisk finisher threads = 3 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: filestore apply finisher threads = 3 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: bdev debug aio = true 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug misdirected ops = true 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore block size = 96636764160 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore compression algorithm = zlib 2024-09-14T07:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore compression mode = aggressive 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore fsck on mount = True 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluefs = 1/20 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluestore = 1/20 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: debug osd = 20 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: debug rocksdb = 4/10 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd full ratio = 0.9 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd nearfull ratio = 0.8 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: osd blocked scrub grace period = 3600 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: osd failsafe full ratio = 0.95 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: osd max markdown count = 1000 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: osd objectstore = bluestore 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.542 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr] 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug mgr = 20 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: mgr/telemetry/nag = false 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout:[mon] 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug paxos = 20 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-14T07:06:54.543 INFO:teuthology.orchestra.run.smithi110.stdout: mon data avail warn = 5 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: mon mgr mkfs grace = 240 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd reporter subtree level = osd 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd prime pg temp = true 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T07:06:54.544 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: # 1m isn't quite enough 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: mon_down_mkfs_grace = 2m 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_filestore_osds = false 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout:[client] 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: rgw cache enabled = true 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable ops log = true 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable usage log = true 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.a] 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.c] 2024-09-14T07:06:54.545 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.b] 2024-09-14T07:06:54.547 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T07:06:54.548 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T07:06:54.638 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T07:06:54.639 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:54.639 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T07:06:54.644 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:54.644 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T07:06:54.692 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:06:54.692 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:54.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T07:06:54.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T07:06:54.755 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:06:54.755 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:06:54.803 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:06:54.804 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:06:54.804 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T07:06:54.804 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T07:06:54.827 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:06:54.827 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:06:54.876 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi145.front.sepia.ceph.com 2024-09-14T07:06:54.876 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:06:54.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T07:06:54.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T07:06:54.899 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:06:54.900 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:06:54.947 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T07:06:54.947 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T07:06:54.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T07:06:54.990 INFO:teuthology.orchestra.run.smithi110.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T07:06:55.005 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T07:06:55.005 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T07:06:55.005 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T07:06:55.041 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T07:06:55.070 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T07:06:55.070 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-14T07:06:55.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T07:06:55.084 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-14T07:06:55.085 INFO:tasks.ceph:role: osd.0 2024-09-14T07:06:55.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:06:55.085 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T07:06:55.141 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T07:06:55.165 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-09-14T07:06:55.167 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-09-14T07:06:55.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T07:06:55.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T07:06:55.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:06:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T07:06:55.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T07:06:55.333 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-14T07:06:55.333 INFO:tasks.ceph:role: osd.1 2024-09-14T07:06:55.333 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:06:55.333 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T07:06:55.387 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T07:06:55.388 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T07:06:55.411 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-09-14T07:06:55.414 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-09-14T07:06:55.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T07:06:55.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T07:06:55.524 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T07:06:55.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:06:55.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T07:06:55.577 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-14T07:06:55.577 INFO:tasks.ceph:role: osd.2 2024-09-14T07:06:55.577 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:06:55.577 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T07:06:55.632 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T07:06:55.633 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T07:06:55.657 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-09-14T07:06:55.658 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-09-14T07:06:55.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T07:06:55.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T07:06:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T07:06:55.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:06:55.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:06:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:55.887+0000 7fceed0337c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T07:06:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:55.887+0000 7fceed0337c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T07:06:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:55.887+0000 7fceed0337c0 -1 bdev(0x560456667800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T07:06:55.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:55.887+0000 7fceed0337c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T07:06:59.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T07:06:59.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:06:59.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:59.135+0000 7fea1d9af7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T07:06:59.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:59.135+0000 7fea1d9af7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T07:06:59.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:59.135+0000 7fea1d9af7c0 -1 bdev(0x55e5c7641800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T07:06:59.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:06:59.135+0000 7fea1d9af7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T07:07:02.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T07:07:02.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:07:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:07:02.419+0000 7fc2af99c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T07:07:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:07:02.419+0000 7fc2af99c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T07:07:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:07:02.423+0000 7fc2af99c7c0 -1 bdev(0x564a87381800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T07:07:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:07:02.423+0000 7fc2af99c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T07:07:05.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T07:07:05.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T07:07:05.668 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-14T07:07:05.668 INFO:tasks.ceph:role: osd.3 2024-09-14T07:07:05.668 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:07:05.669 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T07:07:05.725 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T07:07:05.749 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-14T07:07:05.751 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-14T07:07:05.751 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-14T07:07:05.808 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T07:07:05.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:07:05.860 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T07:07:05.861 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T07:07:05.914 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-14T07:07:05.914 INFO:tasks.ceph:role: osd.4 2024-09-14T07:07:05.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:07:05.914 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T07:07:05.967 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T07:07:05.991 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-14T07:07:05.993 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-14T07:07:05.993 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-14T07:07:06.053 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T07:07:06.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:07:06.105 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T07:07:06.105 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T07:07:06.158 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-14T07:07:06.158 INFO:tasks.ceph:role: osd.5 2024-09-14T07:07:06.158 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:07:06.158 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T07:07:06.211 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T07:07:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T07:07:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T07:07:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-14T07:07:06.212 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T07:07:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-14T07:07:06.212 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T07:07:06.212 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-14T07:07:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T07:07:06.212 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T07:07:06.235 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-14T07:07:06.236 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-14T07:07:06.237 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-14T07:07:06.301 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T07:07:06.353 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: /sbin/restorecon: command not found 2024-09-14T07:07:06.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:07:06.354 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:07:06.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:06.462+0000 7f019677a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T07:07:06.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:06.462+0000 7f019677a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T07:07:06.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:06.462+0000 7f019677a7c0 -1 bdev(0x5555a6b55800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T07:07:06.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:06.462+0000 7f019677a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T07:07:09.593 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T07:07:09.608 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:07:09.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:09.686+0000 7f7a6dc867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T07:07:09.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:09.686+0000 7f7a6dc867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T07:07:09.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:09.686+0000 7f7a6dc867c0 -1 bdev(0x556816c21800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T07:07:09.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:09.686+0000 7f7a6dc867c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T07:07:12.852 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T07:07:12.908 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:07:12.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:12.986+0000 7fe1017bc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T07:07:12.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:12.986+0000 7fe1017bc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T07:07:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:12.986+0000 7fe1017bc7c0 -1 bdev(0x55b3055b7800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T07:07:12.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-14T07:07:12.986+0000 7fe1017bc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T07:07:16.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T07:07:16.164 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T07:07:16.165 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:07:16.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T07:07:16.180 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:07:16.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T07:07:16.236 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:07:16.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T07:07:16.291 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:07:16.291 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T07:07:16.305 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:07:16.305 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T07:07:16.360 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:07:16.360 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T07:07:16.415 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:07:16.415 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T07:07:16.470 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-09-14T07:07:16.470 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T07:07:16.476 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T07:07:16.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T07:07:16.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAPNuVmGWb7NBAA5I3RNtxQN9J+fiRZxz7PvQ== 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQATNuVmE0weCBAAY/xynjtm0umfyMYXGukOVA== 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAWNuVm42BDGRAACL/P0+QMXKxuy8CpyTwkPw== 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2024-09-14T07:07:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAONuVmp7YPOxAAMibvrqH59n5l/eJ4IXEpdA== 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAaNuVmEce+GxAAy0VALIsdxWiUe628fI7Jlg== 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAdNuVmmo43KRAAO4wcQ2zfhI9RUpkKZuIIGw== 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAgNuVmk2gOOxAAxHIeOmkLw9VCDipkddwACQ== 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-09-14T07:07:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAPNuVm7wiRAhAA/6VD7VttGJ5af8kyU6vw3A== 2024-09-14T07:07:16.521 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.0] 2024-09-14T07:07:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAPNuVmGWb7NBAA5I3RNtxQN9J+fiRZxz7PvQ== 2024-09-14T07:07:16.521 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.1] 2024-09-14T07:07:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQATNuVmE0weCBAAY/xynjtm0umfyMYXGukOVA== 2024-09-14T07:07:16.521 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.2] 2024-09-14T07:07:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAWNuVm42BDGRAACL/P0+QMXKxuy8CpyTwkPw== 2024-09-14T07:07:16.521 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.x] 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAONuVmp7YPOxAAMibvrqH59n5l/eJ4IXEpdA== 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.3] 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAaNuVmEce+GxAAy0VALIsdxWiUe628fI7Jlg== 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.4] 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAdNuVmmo43KRAAO4wcQ2zfhI9RUpkKZuIIGw== 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.5] 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAgNuVmk2gOOxAAxHIeOmkLw9VCDipkddwACQ== 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-14T07:07:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAPNuVm7wiRAhAA/6VD7VttGJ5af8kyU6vw3A== 2024-09-14T07:07:16.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.615 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.704 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.748 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T07:07:16.788 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T07:07:16.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.876 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:16.960 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:17.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:17.006 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T07:07:17.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T07:07:17.090 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T07:07:17.174 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T07:07:17.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T07:07:17.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T07:07:17.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T07:07:17.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T07:07:17.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T07:07:17.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T07:07:17.483 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T07:07:17.497 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T07:07:17.590 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T07:07:17.602 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:07:17.604 DEBUG:teuthology.orchestra.run.smithi110:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T07:07:17.648 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T07:07:17.648 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T07:07:17.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T07:07:17.652 INFO:tasks.ceph.mon.a:Started 2024-09-14T07:07:17.652 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T07:07:17.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T07:07:17.655 INFO:tasks.ceph.mon.c:Started 2024-09-14T07:07:17.655 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T07:07:17.655 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T07:07:17.693 INFO:tasks.ceph.mon.b:Started 2024-09-14T07:07:17.693 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T07:07:17.693 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T07:07:17.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T07:07:17.696 INFO:tasks.ceph.mgr.x:Started 2024-09-14T07:07:17.696 DEBUG:tasks.ceph:set 0 configs 2024-09-14T07:07:17.696 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T07:07:17.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T07:07:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-09-14T07:07:18.150 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T07:07:18.150 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T07:07:18.151 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:07:18.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T07:07:18.163 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:07:18.163 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T07:07:18.218 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-14T07:07:18.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T07:07:18.271 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:07:18.271 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T07:07:18.285 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:07:18.285 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T07:07:18.339 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-14T07:07:18.339 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T07:07:18.394 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 9c97eae7-11c5-4962-a62d-04d698e50114 0 2024-09-14T07:07:18.644 INFO:teuthology.orchestra.run.smithi110.stdout:0 2024-09-14T07:07:18.662 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 91d33ac7-8429-4237-bb93-1259b318642a 1 2024-09-14T07:07:18.863 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-14T07:07:18.880 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 02202e5a-e516-4438-9378-7fad80297e64 2 2024-09-14T07:07:19.085 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-14T07:07:19.101 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 57a45940-90af-484d-9570-8de1c6b2252b 3 2024-09-14T07:07:19.299 INFO:teuthology.orchestra.run.smithi110.stdout:3 2024-09-14T07:07:19.315 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 8150dd96-bfa0-404f-ae37-f18897bcc93e 4 2024-09-14T07:07:19.503 INFO:teuthology.orchestra.run.smithi110.stdout:4 2024-09-14T07:07:19.519 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 59d63d5d-8ef9-43b2-8d7f-49e8e8385435 5 2024-09-14T07:07:19.715 INFO:teuthology.orchestra.run.smithi110.stdout:5 2024-09-14T07:07:19.731 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T07:07:19.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T07:07:19.734 INFO:tasks.ceph.osd.0:Started 2024-09-14T07:07:19.734 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T07:07:19.734 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T07:07:19.736 INFO:tasks.ceph.osd.1:Started 2024-09-14T07:07:19.736 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T07:07:19.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T07:07:19.739 INFO:tasks.ceph.osd.2:Started 2024-09-14T07:07:19.739 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T07:07:19.739 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T07:07:19.741 INFO:tasks.ceph.osd.3:Started 2024-09-14T07:07:19.741 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T07:07:19.741 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T07:07:19.743 INFO:tasks.ceph.osd.4:Started 2024-09-14T07:07:19.743 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T07:07:19.743 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T07:07:19.745 INFO:tasks.ceph.osd.5:Started 2024-09-14T07:07:19.745 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:07:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:19.938 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:07:19.712761+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T07:07:19.712761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:07:19.956 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T07:07:19.956 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T07:07:20.560 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T07:07:20.768 INFO:teuthology.misc.health.smithi005.stdout: 2024-09-14T07:07:20.768 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":8,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:07:19.712761+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T07:07:19.712761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:07:20.784 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-14T07:07:20.844 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:20.843+0000 7f68a4baf7c0 -1 Falling back to public interface 2024-09-14T07:07:20.851 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:20.851+0000 7fa9014947c0 -1 Falling back to public interface 2024-09-14T07:07:20.885 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:20.882+0000 7fa13237b7c0 -1 Falling back to public interface 2024-09-14T07:07:20.925 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:20.922+0000 7f97cd36e7c0 -1 Falling back to public interface 2024-09-14T07:07:21.101 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:21.099+0000 7f7d137297c0 -1 Falling back to public interface 2024-09-14T07:07:21.282 INFO:tasks.ceph.mgr.x.smithi110.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-14T07:07:21.282 INFO:tasks.ceph.mgr.x.smithi110.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-14T07:07:21.282 INFO:tasks.ceph.mgr.x.smithi110.stderr: from numpy import show_config as show_numpy_config 2024-09-14T07:07:21.353 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:21.350+0000 7f8d23d207c0 -1 Falling back to public interface 2024-09-14T07:07:24.209 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-14T07:07:24.206+0000 7f48973a8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T07:07:24.209 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-14T07:07:24.206+0000 7f48973a8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T07:07:24.690 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:24.686+0000 7f97cd36e7c0 -1 osd.5 0 log_to_monitors true 2024-09-14T07:07:24.821 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:24.819+0000 7fa9014947c0 -1 osd.2 0 log_to_monitors true 2024-09-14T07:07:24.919 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:24.914+0000 7fa13237b7c0 -1 osd.3 0 log_to_monitors true 2024-09-14T07:07:24.992 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:24.991+0000 7f7d137297c0 -1 osd.0 0 log_to_monitors true 2024-09-14T07:07:25.097 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:25.095+0000 7f68a4baf7c0 -1 osd.1 0 log_to_monitors true 2024-09-14T07:07:25.336 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:25.334+0000 7f8d23d207c0 -1 osd.4 0 log_to_monitors true 2024-09-14T07:07:27.391 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T07:07:27.715 INFO:teuthology.misc.health.smithi005.stdout: 2024-09-14T07:07:27.715 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":11,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:07:27.268818+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T07:07:19.712761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:07:27.732 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-14T07:07:29.203 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:29.202+0000 7fa12e324640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T07:07:29.232 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:29.230+0000 7fa126156640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T07:07:29.291 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:29.286+0000 7f97c9317640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T07:07:29.324 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:29.322+0000 7f97c1149640 -1 osd.5 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T07:07:29.376 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:29.375+0000 7f68a136a640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T07:07:29.391 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:29.391+0000 7f7d0f6d2640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T07:07:29.399 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:29.399+0000 7f689898a640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T07:07:29.408 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:29.407+0000 7fa8fdc4f640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T07:07:29.415 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:29.415+0000 7f7d07504640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T07:07:29.439 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:29.439+0000 7fa8f526f640 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T07:07:30.241 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:30.238+0000 7f8d204db640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T07:07:30.277 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:30.274+0000 7f8d17afb640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T07:07:34.338 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T07:07:34.648 INFO:teuthology.misc.health.smithi005.stdout: 2024-09-14T07:07:34.648 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":15,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:07:33.293961+0000","last_up_change":"2024-09-14T07:07:31.282745+0000","last_in_change":"2024-09-14T07:07:19.712761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T07:07:31.246122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6817","nonce":1673964337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6819","nonce":1673964337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6823","nonce":1673964337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6821","nonce":1673964337}]},"public_addr":"172.21.15.5:6817/1673964337","cluster_addr":"172.21.15.5:6819/1673964337","heartbeat_back_addr":"172.21.15.5:6823/1673964337","heartbeat_front_addr":"172.21.15.5:6821/1673964337","state":["exists","up"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6801","nonce":1704459401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6803","nonce":1704459401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6807","nonce":1704459401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6805","nonce":1704459401}]},"public_addr":"172.21.15.5:6801/1704459401","cluster_addr":"172.21.15.5:6803/1704459401","heartbeat_back_addr":"172.21.15.5:6807/1704459401","heartbeat_front_addr":"172.21.15.5:6805/1704459401","state":["exists","up"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6809","nonce":2565128645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6811","nonce":2565128645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6815","nonce":2565128645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6813","nonce":2565128645}]},"public_addr":"172.21.15.5:6809/2565128645","cluster_addr":"172.21.15.5:6811/2565128645","heartbeat_back_addr":"172.21.15.5:6815/2565128645","heartbeat_front_addr":"172.21.15.5:6813/2565128645","state":["exists","up"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6801","nonce":2226792373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6803","nonce":2226792373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6807","nonce":2226792373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6805","nonce":2226792373}]},"public_addr":"172.21.15.110:6801/2226792373","cluster_addr":"172.21.15.110:6803/2226792373","heartbeat_back_addr":"172.21.15.110:6807/2226792373","heartbeat_front_addr":"172.21.15.110:6805/2226792373","state":["exists","up"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6817","nonce":940255113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6819","nonce":940255113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6823","nonce":940255113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6821","nonce":940255113}]},"public_addr":"172.21.15.110:6817/940255113","cluster_addr":"172.21.15.110:6819/940255113","heartbeat_back_addr":"172.21.15.110:6823/940255113","heartbeat_front_addr":"172.21.15.110:6821/940255113","state":["exists","up"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6809","nonce":4012173203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6811","nonce":4012173203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6815","nonce":4012173203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6813","nonce":4012173203}]},"public_addr":"172.21.15.110:6809/4012173203","cluster_addr":"172.21.15.110:6811/4012173203","heartbeat_back_addr":"172.21.15.110:6815/4012173203","heartbeat_front_addr":"172.21.15.110:6813/4012173203","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.952391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.101881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.869317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.961339+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.296733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.723820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:07:34.666 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-14T07:07:34.666 INFO:tasks.ceph:Creating RBD pool 2024-09-14T07:07:34.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T07:07:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'rbd' created 2024-09-14T07:07:35.016 DEBUG:teuthology.orchestra.run.smithi005:> rbd --cluster ceph pool init rbd 2024-09-14T07:07:38.048 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T07:07:38.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-14T07:07:38.049 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T07:07:38.371 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:38.391 INFO:teuthology.orchestra.run.smithi005.stdout:[{"version":7,"timestamp":"2024-09-14T07:07:30.243594+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"6172.402282"}]}] 2024-09-14T07:07:38.391 INFO:tasks.ceph_manager:config epoch is 7 2024-09-14T07:07:38.392 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T07:07:38.392 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T07:07:38.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T07:07:38.745 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:38.762 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":2233198563},{"type":"v1","addr":"172.21.15.110:6825","nonce":2233198563}]},"active_addr":"172.21.15.110:6825/2233198563","active_change":"2024-09-14T07:07:23.192452+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":4255979761}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3681722906}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":2748575678}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1037975402}]}]} 2024-09-14T07:07:38.765 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T07:07:38.765 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T07:07:38.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:07:39.069 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:39.070 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":19,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:07:38.019705+0000","last_up_change":"2024-09-14T07:07:31.282745+0000","last_in_change":"2024-09-14T07:07:19.712761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T07:07:31.246122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T07:07:34.989638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6817","nonce":1673964337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6819","nonce":1673964337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6823","nonce":1673964337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6821","nonce":1673964337}]},"public_addr":"172.21.15.5:6817/1673964337","cluster_addr":"172.21.15.5:6819/1673964337","heartbeat_back_addr":"172.21.15.5:6823/1673964337","heartbeat_front_addr":"172.21.15.5:6821/1673964337","state":["exists","up"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6801","nonce":1704459401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6803","nonce":1704459401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6807","nonce":1704459401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6805","nonce":1704459401}]},"public_addr":"172.21.15.5:6801/1704459401","cluster_addr":"172.21.15.5:6803/1704459401","heartbeat_back_addr":"172.21.15.5:6807/1704459401","heartbeat_front_addr":"172.21.15.5:6805/1704459401","state":["exists","up"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6809","nonce":2565128645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6811","nonce":2565128645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6815","nonce":2565128645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6813","nonce":2565128645}]},"public_addr":"172.21.15.5:6809/2565128645","cluster_addr":"172.21.15.5:6811/2565128645","heartbeat_back_addr":"172.21.15.5:6815/2565128645","heartbeat_front_addr":"172.21.15.5:6813/2565128645","state":["exists","up"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6801","nonce":2226792373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6803","nonce":2226792373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6807","nonce":2226792373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6805","nonce":2226792373}]},"public_addr":"172.21.15.110:6801/2226792373","cluster_addr":"172.21.15.110:6803/2226792373","heartbeat_back_addr":"172.21.15.110:6807/2226792373","heartbeat_front_addr":"172.21.15.110:6805/2226792373","state":["exists","up"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6817","nonce":940255113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6819","nonce":940255113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6823","nonce":940255113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6821","nonce":940255113}]},"public_addr":"172.21.15.110:6817/940255113","cluster_addr":"172.21.15.110:6819/940255113","heartbeat_back_addr":"172.21.15.110:6823/940255113","heartbeat_front_addr":"172.21.15.110:6821/940255113","state":["exists","up"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6809","nonce":4012173203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6811","nonce":4012173203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6815","nonce":4012173203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6813","nonce":4012173203}]},"public_addr":"172.21.15.110:6809/4012173203","cluster_addr":"172.21.15.110:6811/4012173203","heartbeat_back_addr":"172.21.15.110:6815/4012173203","heartbeat_front_addr":"172.21.15.110:6813/4012173203","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.952391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.101881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.869317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.961339+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.296733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.723820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:07:39.087 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T07:07:39.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:07:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:39.374 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":19,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:07:38.019705+0000","last_up_change":"2024-09-14T07:07:31.282745+0000","last_in_change":"2024-09-14T07:07:19.712761+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T07:07:31.246122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T07:07:34.989638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6817","nonce":1673964337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6819","nonce":1673964337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6823","nonce":1673964337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1673964337},{"type":"v1","addr":"172.21.15.5:6821","nonce":1673964337}]},"public_addr":"172.21.15.5:6817/1673964337","cluster_addr":"172.21.15.5:6819/1673964337","heartbeat_back_addr":"172.21.15.5:6823/1673964337","heartbeat_front_addr":"172.21.15.5:6821/1673964337","state":["exists","up"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6801","nonce":1704459401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6803","nonce":1704459401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6807","nonce":1704459401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6805","nonce":1704459401}]},"public_addr":"172.21.15.5:6801/1704459401","cluster_addr":"172.21.15.5:6803/1704459401","heartbeat_back_addr":"172.21.15.5:6807/1704459401","heartbeat_front_addr":"172.21.15.5:6805/1704459401","state":["exists","up"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6809","nonce":2565128645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6811","nonce":2565128645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6815","nonce":2565128645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6813","nonce":2565128645}]},"public_addr":"172.21.15.5:6809/2565128645","cluster_addr":"172.21.15.5:6811/2565128645","heartbeat_back_addr":"172.21.15.5:6815/2565128645","heartbeat_front_addr":"172.21.15.5:6813/2565128645","state":["exists","up"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6801","nonce":2226792373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6803","nonce":2226792373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6807","nonce":2226792373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6805","nonce":2226792373}]},"public_addr":"172.21.15.110:6801/2226792373","cluster_addr":"172.21.15.110:6803/2226792373","heartbeat_back_addr":"172.21.15.110:6807/2226792373","heartbeat_front_addr":"172.21.15.110:6805/2226792373","state":["exists","up"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6817","nonce":940255113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6819","nonce":940255113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6823","nonce":940255113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6821","nonce":940255113}]},"public_addr":"172.21.15.110:6817/940255113","cluster_addr":"172.21.15.110:6819/940255113","heartbeat_back_addr":"172.21.15.110:6823/940255113","heartbeat_front_addr":"172.21.15.110:6821/940255113","state":["exists","up"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6809","nonce":4012173203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6811","nonce":4012173203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6815","nonce":4012173203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6813","nonce":4012173203}]},"public_addr":"172.21.15.110:6809/4012173203","cluster_addr":"172.21.15.110:6811/4012173203","heartbeat_back_addr":"172.21.15.110:6815/4012173203","heartbeat_front_addr":"172.21.15.110:6813/4012173203","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.952391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.101881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.869317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.961339+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.296733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.723820+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:07:39.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T07:07:39.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T07:07:39.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T07:07:39.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T07:07:39.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T07:07:39.394 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T07:07:39.558 INFO:teuthology.orchestra.run.smithi005.stdout:51539607558 2024-09-14T07:07:39.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T07:07:39.570 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:39.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T07:07:39.622 INFO:teuthology.orchestra.run.smithi005.stdout:51539607556 2024-09-14T07:07:39.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T07:07:39.630 INFO:teuthology.orchestra.run.smithi005.stdout:51539607558 2024-09-14T07:07:39.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T07:07:39.638 INFO:teuthology.orchestra.run.smithi005.stdout:51539607558 2024-09-14T07:07:39.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T07:07:39.644 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-09-14T07:07:39.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T07:07:40.009 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:40.020 INFO:teuthology.orchestra.run.smithi005.stdout:55834574854 2024-09-14T07:07:40.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.2 2024-09-14T07:07:40.027 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:40.028 INFO:teuthology.orchestra.run.smithi005.stdout:51539607555 2024-09-14T07:07:40.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.4 2024-09-14T07:07:40.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.3 2024-09-14T07:07:40.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.1 2024-09-14T07:07:40.080 INFO:teuthology.orchestra.run.smithi005.stdout:51539607556 2024-09-14T07:07:40.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.5 2024-09-14T07:07:40.138 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:40.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.0 2024-09-14T07:07:41.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T07:07:41.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T07:07:41.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T07:07:41.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T07:07:41.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T07:07:41.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T07:07:41.401 INFO:teuthology.orchestra.run.smithi005.stdout:55834574855 2024-09-14T07:07:41.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-09-14T07:07:41.425 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:41.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.2 2024-09-14T07:07:41.480 INFO:teuthology.orchestra.run.smithi005.stdout:51539607555 2024-09-14T07:07:41.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.3 2024-09-14T07:07:41.532 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:41.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.1 2024-09-14T07:07:41.570 INFO:teuthology.orchestra.run.smithi005.stdout:51539607556 2024-09-14T07:07:41.582 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:41.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.5 2024-09-14T07:07:41.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.0 2024-09-14T07:07:42.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T07:07:42.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T07:07:42.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T07:07:42.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T07:07:42.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T07:07:42.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T07:07:42.833 INFO:teuthology.orchestra.run.smithi005.stdout:51539607559 2024-09-14T07:07:42.856 INFO:teuthology.orchestra.run.smithi005.stdout:55834574856 2024-09-14T07:07:42.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607559 for osd.2 2024-09-14T07:07:42.859 DEBUG:teuthology.parallel:result is None 2024-09-14T07:07:42.872 INFO:teuthology.orchestra.run.smithi005.stdout:51539607557 2024-09-14T07:07:42.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-09-14T07:07:42.883 DEBUG:teuthology.parallel:result is None 2024-09-14T07:07:42.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607557 for osd.3 2024-09-14T07:07:42.891 DEBUG:teuthology.parallel:result is None 2024-09-14T07:07:42.919 INFO:teuthology.orchestra.run.smithi005.stdout:51539607559 2024-09-14T07:07:42.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607559 for osd.0 2024-09-14T07:07:42.937 DEBUG:teuthology.parallel:result is None 2024-09-14T07:07:42.962 INFO:teuthology.orchestra.run.smithi005.stdout:51539607559 2024-09-14T07:07:42.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607559 for osd.1 2024-09-14T07:07:42.979 DEBUG:teuthology.parallel:result is None 2024-09-14T07:07:43.006 INFO:teuthology.orchestra.run.smithi005.stdout:51539607558 2024-09-14T07:07:43.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607558 for osd.5 2024-09-14T07:07:43.023 DEBUG:teuthology.parallel:result is None 2024-09-14T07:07:43.023 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T07:07:43.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:07:43.362 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:43.363 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:07:43.379 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T07:07:43.206306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":188328,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566042712,"statfs":{"total":579820584960,"available":579627737088,"internally_reserved":0,"allocated":1204224,"data_stored":1537378,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.186760"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030798+0000","last_change":"2024-09-14T07:07:38.031294+0000","last_active":"2024-09-14T07:07:38.030798+0000","last_peered":"2024-09-14T07:07:38.030798+0000","last_clean":"2024-09-14T07:07:38.030798+0000","last_became_active":"2024-09-14T07:07:36.418930+0000","last_became_peered":"2024-09-14T07:07:36.418930+0000","last_unstale":"2024-09-14T07:07:38.030798+0000","last_undegraded":"2024-09-14T07:07:38.030798+0000","last_fullsized":"2024-09-14T07:07:38.030798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:08.172090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031770+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:07:38.031770+0000","last_peered":"2024-09-14T07:07:38.031770+0000","last_clean":"2024-09-14T07:07:38.031770+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:07:38.031770+0000","last_undegraded":"2024-09-14T07:07:38.031770+0000","last_fullsized":"2024-09-14T07:07:38.031770+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030704+0000","last_change":"2024-09-14T07:07:38.031148+0000","last_active":"2024-09-14T07:07:38.030704+0000","last_peered":"2024-09-14T07:07:38.030704+0000","last_clean":"2024-09-14T07:07:38.030704+0000","last_became_active":"2024-09-14T07:07:36.196549+0000","last_became_peered":"2024-09-14T07:07:36.196549+0000","last_unstale":"2024-09-14T07:07:38.030704+0000","last_undegraded":"2024-09-14T07:07:38.030704+0000","last_fullsized":"2024-09-14T07:07:38.030704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:35.911258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031619+0000","last_change":"2024-09-14T07:07:38.032270+0000","last_active":"2024-09-14T07:07:38.031619+0000","last_peered":"2024-09-14T07:07:38.031619+0000","last_clean":"2024-09-14T07:07:38.031619+0000","last_became_active":"2024-09-14T07:07:36.196923+0000","last_became_peered":"2024-09-14T07:07:36.196923+0000","last_unstale":"2024-09-14T07:07:38.031619+0000","last_undegraded":"2024-09-14T07:07:38.031619+0000","last_fullsized":"2024-09-14T07:07:38.031619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:20.173745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.039482+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:07:38.039482+0000","last_peered":"2024-09-14T07:07:38.039482+0000","last_clean":"2024-09-14T07:07:38.039482+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:07:38.039482+0000","last_undegraded":"2024-09-14T07:07:38.039482+0000","last_fullsized":"2024-09-14T07:07:38.039482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.097746+0000","last_change":"2024-09-14T07:07:38.097746+0000","last_active":"2024-09-14T07:07:38.097746+0000","last_peered":"2024-09-14T07:07:38.097746+0000","last_clean":"2024-09-14T07:07:38.097746+0000","last_became_active":"2024-09-14T07:07:36.018077+0000","last_became_peered":"2024-09-14T07:07:36.018077+0000","last_unstale":"2024-09-14T07:07:38.097746+0000","last_undegraded":"2024-09-14T07:07:38.097746+0000","last_fullsized":"2024-09-14T07:07:38.097746+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:46:13.853552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068874500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031291+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:07:38.031291+0000","last_peered":"2024-09-14T07:07:38.031291+0000","last_clean":"2024-09-14T07:07:38.031291+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:07:38.031291+0000","last_undegraded":"2024-09-14T07:07:38.031291+0000","last_fullsized":"2024-09-14T07:07:38.031291+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.036940+0000","last_change":"2024-09-14T07:07:38.036940+0000","last_active":"2024-09-14T07:07:38.036940+0000","last_peered":"2024-09-14T07:07:38.036940+0000","last_clean":"2024-09-14T07:07:38.036940+0000","last_became_active":"2024-09-14T07:07:36.012879+0000","last_became_peered":"2024-09-14T07:07:36.012879+0000","last_unstale":"2024-09-14T07:07:38.036940+0000","last_undegraded":"2024-09-14T07:07:38.036940+0000","last_fullsized":"2024-09-14T07:07:38.036940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:42.183903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.029931+0000","last_change":"2024-09-14T07:07:32.604869+0000","last_active":"2024-09-14T07:07:38.029931+0000","last_peered":"2024-09-14T07:07:38.029931+0000","last_clean":"2024-09-14T07:07:38.029931+0000","last_became_active":"2024-09-14T07:07:32.602241+0000","last_became_peered":"2024-09-14T07:07:32.602241+0000","last_unstale":"2024-09-14T07:07:38.029931+0000","last_undegraded":"2024-09-14T07:07:38.029931+0000","last_fullsized":"2024-09-14T07:07:38.029931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:25.905190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607559,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:07:43.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:07:43.664 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:07:43.681 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T07:07:43.206306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":188328,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566042712,"statfs":{"total":579820584960,"available":579627737088,"internally_reserved":0,"allocated":1204224,"data_stored":1537378,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.186760"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030798+0000","last_change":"2024-09-14T07:07:38.031294+0000","last_active":"2024-09-14T07:07:38.030798+0000","last_peered":"2024-09-14T07:07:38.030798+0000","last_clean":"2024-09-14T07:07:38.030798+0000","last_became_active":"2024-09-14T07:07:36.418930+0000","last_became_peered":"2024-09-14T07:07:36.418930+0000","last_unstale":"2024-09-14T07:07:38.030798+0000","last_undegraded":"2024-09-14T07:07:38.030798+0000","last_fullsized":"2024-09-14T07:07:38.030798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:08.172090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031770+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:07:38.031770+0000","last_peered":"2024-09-14T07:07:38.031770+0000","last_clean":"2024-09-14T07:07:38.031770+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:07:38.031770+0000","last_undegraded":"2024-09-14T07:07:38.031770+0000","last_fullsized":"2024-09-14T07:07:38.031770+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030704+0000","last_change":"2024-09-14T07:07:38.031148+0000","last_active":"2024-09-14T07:07:38.030704+0000","last_peered":"2024-09-14T07:07:38.030704+0000","last_clean":"2024-09-14T07:07:38.030704+0000","last_became_active":"2024-09-14T07:07:36.196549+0000","last_became_peered":"2024-09-14T07:07:36.196549+0000","last_unstale":"2024-09-14T07:07:38.030704+0000","last_undegraded":"2024-09-14T07:07:38.030704+0000","last_fullsized":"2024-09-14T07:07:38.030704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:35.911258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031619+0000","last_change":"2024-09-14T07:07:38.032270+0000","last_active":"2024-09-14T07:07:38.031619+0000","last_peered":"2024-09-14T07:07:38.031619+0000","last_clean":"2024-09-14T07:07:38.031619+0000","last_became_active":"2024-09-14T07:07:36.196923+0000","last_became_peered":"2024-09-14T07:07:36.196923+0000","last_unstale":"2024-09-14T07:07:38.031619+0000","last_undegraded":"2024-09-14T07:07:38.031619+0000","last_fullsized":"2024-09-14T07:07:38.031619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:20.173745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.039482+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:07:38.039482+0000","last_peered":"2024-09-14T07:07:38.039482+0000","last_clean":"2024-09-14T07:07:38.039482+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:07:38.039482+0000","last_undegraded":"2024-09-14T07:07:38.039482+0000","last_fullsized":"2024-09-14T07:07:38.039482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.097746+0000","last_change":"2024-09-14T07:07:38.097746+0000","last_active":"2024-09-14T07:07:38.097746+0000","last_peered":"2024-09-14T07:07:38.097746+0000","last_clean":"2024-09-14T07:07:38.097746+0000","last_became_active":"2024-09-14T07:07:36.018077+0000","last_became_peered":"2024-09-14T07:07:36.018077+0000","last_unstale":"2024-09-14T07:07:38.097746+0000","last_undegraded":"2024-09-14T07:07:38.097746+0000","last_fullsized":"2024-09-14T07:07:38.097746+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:46:13.853552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068874500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031291+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:07:38.031291+0000","last_peered":"2024-09-14T07:07:38.031291+0000","last_clean":"2024-09-14T07:07:38.031291+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:07:38.031291+0000","last_undegraded":"2024-09-14T07:07:38.031291+0000","last_fullsized":"2024-09-14T07:07:38.031291+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.036940+0000","last_change":"2024-09-14T07:07:38.036940+0000","last_active":"2024-09-14T07:07:38.036940+0000","last_peered":"2024-09-14T07:07:38.036940+0000","last_clean":"2024-09-14T07:07:38.036940+0000","last_became_active":"2024-09-14T07:07:36.012879+0000","last_became_peered":"2024-09-14T07:07:36.012879+0000","last_unstale":"2024-09-14T07:07:38.036940+0000","last_undegraded":"2024-09-14T07:07:38.036940+0000","last_fullsized":"2024-09-14T07:07:38.036940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:42.183903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.029931+0000","last_change":"2024-09-14T07:07:32.604869+0000","last_active":"2024-09-14T07:07:38.029931+0000","last_peered":"2024-09-14T07:07:38.029931+0000","last_clean":"2024-09-14T07:07:38.029931+0000","last_became_active":"2024-09-14T07:07:32.602241+0000","last_became_peered":"2024-09-14T07:07:32.602241+0000","last_unstale":"2024-09-14T07:07:38.029931+0000","last_undegraded":"2024-09-14T07:07:38.029931+0000","last_fullsized":"2024-09-14T07:07:38.029931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:25.905190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607559,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:07:43.682 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T07:07:43.682 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T07:07:43.682 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T07:07:43.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T07:07:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:44.007 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T07:07:44.024 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T07:07:44.025 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T07:07:44.037 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T07:07:44.037 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}} 2024-09-14T07:07:44.037 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T07:07:44.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T07:07:44.361 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T07:07:44.361 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T07:07:44.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:07:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:07:44.672 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T07:07:43.206306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":188328,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566042712,"statfs":{"total":579820584960,"available":579627737088,"internally_reserved":0,"allocated":1204224,"data_stored":1537378,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.186760"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030798+0000","last_change":"2024-09-14T07:07:38.031294+0000","last_active":"2024-09-14T07:07:38.030798+0000","last_peered":"2024-09-14T07:07:38.030798+0000","last_clean":"2024-09-14T07:07:38.030798+0000","last_became_active":"2024-09-14T07:07:36.418930+0000","last_became_peered":"2024-09-14T07:07:36.418930+0000","last_unstale":"2024-09-14T07:07:38.030798+0000","last_undegraded":"2024-09-14T07:07:38.030798+0000","last_fullsized":"2024-09-14T07:07:38.030798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:08.172090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031770+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:07:38.031770+0000","last_peered":"2024-09-14T07:07:38.031770+0000","last_clean":"2024-09-14T07:07:38.031770+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:07:38.031770+0000","last_undegraded":"2024-09-14T07:07:38.031770+0000","last_fullsized":"2024-09-14T07:07:38.031770+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030704+0000","last_change":"2024-09-14T07:07:38.031148+0000","last_active":"2024-09-14T07:07:38.030704+0000","last_peered":"2024-09-14T07:07:38.030704+0000","last_clean":"2024-09-14T07:07:38.030704+0000","last_became_active":"2024-09-14T07:07:36.196549+0000","last_became_peered":"2024-09-14T07:07:36.196549+0000","last_unstale":"2024-09-14T07:07:38.030704+0000","last_undegraded":"2024-09-14T07:07:38.030704+0000","last_fullsized":"2024-09-14T07:07:38.030704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:35.911258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031619+0000","last_change":"2024-09-14T07:07:38.032270+0000","last_active":"2024-09-14T07:07:38.031619+0000","last_peered":"2024-09-14T07:07:38.031619+0000","last_clean":"2024-09-14T07:07:38.031619+0000","last_became_active":"2024-09-14T07:07:36.196923+0000","last_became_peered":"2024-09-14T07:07:36.196923+0000","last_unstale":"2024-09-14T07:07:38.031619+0000","last_undegraded":"2024-09-14T07:07:38.031619+0000","last_fullsized":"2024-09-14T07:07:38.031619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:20.173745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.039482+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:07:38.039482+0000","last_peered":"2024-09-14T07:07:38.039482+0000","last_clean":"2024-09-14T07:07:38.039482+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:07:38.039482+0000","last_undegraded":"2024-09-14T07:07:38.039482+0000","last_fullsized":"2024-09-14T07:07:38.039482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.097746+0000","last_change":"2024-09-14T07:07:38.097746+0000","last_active":"2024-09-14T07:07:38.097746+0000","last_peered":"2024-09-14T07:07:38.097746+0000","last_clean":"2024-09-14T07:07:38.097746+0000","last_became_active":"2024-09-14T07:07:36.018077+0000","last_became_peered":"2024-09-14T07:07:36.018077+0000","last_unstale":"2024-09-14T07:07:38.097746+0000","last_undegraded":"2024-09-14T07:07:38.097746+0000","last_fullsized":"2024-09-14T07:07:38.097746+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:46:13.853552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068874500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031291+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:07:38.031291+0000","last_peered":"2024-09-14T07:07:38.031291+0000","last_clean":"2024-09-14T07:07:38.031291+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:07:38.031291+0000","last_undegraded":"2024-09-14T07:07:38.031291+0000","last_fullsized":"2024-09-14T07:07:38.031291+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.036940+0000","last_change":"2024-09-14T07:07:38.036940+0000","last_active":"2024-09-14T07:07:38.036940+0000","last_peered":"2024-09-14T07:07:38.036940+0000","last_clean":"2024-09-14T07:07:38.036940+0000","last_became_active":"2024-09-14T07:07:36.012879+0000","last_became_peered":"2024-09-14T07:07:36.012879+0000","last_unstale":"2024-09-14T07:07:38.036940+0000","last_undegraded":"2024-09-14T07:07:38.036940+0000","last_fullsized":"2024-09-14T07:07:38.036940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:42.183903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.029931+0000","last_change":"2024-09-14T07:07:32.604869+0000","last_active":"2024-09-14T07:07:38.029931+0000","last_peered":"2024-09-14T07:07:38.029931+0000","last_clean":"2024-09-14T07:07:38.029931+0000","last_became_active":"2024-09-14T07:07:32.602241+0000","last_became_peered":"2024-09-14T07:07:32.602241+0000","last_unstale":"2024-09-14T07:07:38.029931+0000","last_undegraded":"2024-09-14T07:07:38.029931+0000","last_fullsized":"2024-09-14T07:07:38.029931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:25.905190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607559,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:07:44.673 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:07:44.964 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:07:44.980 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-14T07:07:43.206306+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":188328,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":566042712,"statfs":{"total":579820584960,"available":579627737088,"internally_reserved":0,"allocated":1204224,"data_stored":1537378,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":191486624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.186760"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030798+0000","last_change":"2024-09-14T07:07:38.031294+0000","last_active":"2024-09-14T07:07:38.030798+0000","last_peered":"2024-09-14T07:07:38.030798+0000","last_clean":"2024-09-14T07:07:38.030798+0000","last_became_active":"2024-09-14T07:07:36.418930+0000","last_became_peered":"2024-09-14T07:07:36.418930+0000","last_unstale":"2024-09-14T07:07:38.030798+0000","last_undegraded":"2024-09-14T07:07:38.030798+0000","last_fullsized":"2024-09-14T07:07:38.030798+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:08.172090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031770+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:07:38.031770+0000","last_peered":"2024-09-14T07:07:38.031770+0000","last_clean":"2024-09-14T07:07:38.031770+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:07:38.031770+0000","last_undegraded":"2024-09-14T07:07:38.031770+0000","last_fullsized":"2024-09-14T07:07:38.031770+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.030704+0000","last_change":"2024-09-14T07:07:38.031148+0000","last_active":"2024-09-14T07:07:38.030704+0000","last_peered":"2024-09-14T07:07:38.030704+0000","last_clean":"2024-09-14T07:07:38.030704+0000","last_became_active":"2024-09-14T07:07:36.196549+0000","last_became_peered":"2024-09-14T07:07:36.196549+0000","last_unstale":"2024-09-14T07:07:38.030704+0000","last_undegraded":"2024-09-14T07:07:38.030704+0000","last_fullsized":"2024-09-14T07:07:38.030704+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:35.911258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031619+0000","last_change":"2024-09-14T07:07:38.032270+0000","last_active":"2024-09-14T07:07:38.031619+0000","last_peered":"2024-09-14T07:07:38.031619+0000","last_clean":"2024-09-14T07:07:38.031619+0000","last_became_active":"2024-09-14T07:07:36.196923+0000","last_became_peered":"2024-09-14T07:07:36.196923+0000","last_unstale":"2024-09-14T07:07:38.031619+0000","last_undegraded":"2024-09-14T07:07:38.031619+0000","last_fullsized":"2024-09-14T07:07:38.031619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:20.173745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.039482+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:07:38.039482+0000","last_peered":"2024-09-14T07:07:38.039482+0000","last_clean":"2024-09-14T07:07:38.039482+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:07:38.039482+0000","last_undegraded":"2024-09-14T07:07:38.039482+0000","last_fullsized":"2024-09-14T07:07:38.039482+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.097746+0000","last_change":"2024-09-14T07:07:38.097746+0000","last_active":"2024-09-14T07:07:38.097746+0000","last_peered":"2024-09-14T07:07:38.097746+0000","last_clean":"2024-09-14T07:07:38.097746+0000","last_became_active":"2024-09-14T07:07:36.018077+0000","last_became_peered":"2024-09-14T07:07:36.018077+0000","last_unstale":"2024-09-14T07:07:38.097746+0000","last_undegraded":"2024-09-14T07:07:38.097746+0000","last_fullsized":"2024-09-14T07:07:38.097746+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:46:13.853552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068874500000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.031291+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:07:38.031291+0000","last_peered":"2024-09-14T07:07:38.031291+0000","last_clean":"2024-09-14T07:07:38.031291+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:07:38.031291+0000","last_undegraded":"2024-09-14T07:07:38.031291+0000","last_fullsized":"2024-09-14T07:07:38.031291+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.036940+0000","last_change":"2024-09-14T07:07:38.036940+0000","last_active":"2024-09-14T07:07:38.036940+0000","last_peered":"2024-09-14T07:07:38.036940+0000","last_clean":"2024-09-14T07:07:38.036940+0000","last_became_active":"2024-09-14T07:07:36.012879+0000","last_became_peered":"2024-09-14T07:07:36.012879+0000","last_unstale":"2024-09-14T07:07:38.036940+0000","last_undegraded":"2024-09-14T07:07:38.036940+0000","last_fullsized":"2024-09-14T07:07:38.036940+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:42.183903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'72","reported_seq":111,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-14T07:07:38.029931+0000","last_change":"2024-09-14T07:07:32.604869+0000","last_active":"2024-09-14T07:07:38.029931+0000","last_peered":"2024-09-14T07:07:38.029931+0000","last_clean":"2024-09-14T07:07:38.029931+0000","last_became_active":"2024-09-14T07:07:32.602241+0000","last_became_peered":"2024-09-14T07:07:32.602241+0000","last_unstale":"2024-09-14T07:07:38.029931+0000","last_undegraded":"2024-09-14T07:07:38.029931+0000","last_fullsized":"2024-09-14T07:07:38.029931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:25.905190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607559,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607559,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607559,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":649802,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:07:44.981 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T07:07:44.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 20 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 717e3012-acdf-42c1-a4e0-d4c94afd9f15 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-14T07:07:18.019909+0000 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-14T07:07:42.743851+0000 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 5 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-09-14T07:07:45.270 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client jewel 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 6 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6816/1673964337,v1:172.21.15.5:6817/1673964337] [v2:172.21.15.5:6818/1673964337,v1:172.21.15.5:6819/1673964337] exists,up 9c97eae7-11c5-4962-a62d-04d698e50114 2024-09-14T07:07:45.271 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6800/1704459401,v1:172.21.15.5:6801/1704459401] [v2:172.21.15.5:6802/1704459401,v1:172.21.15.5:6803/1704459401] exists,up 91d33ac7-8429-4237-bb93-1259b318642a 2024-09-14T07:07:45.272 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6808/2565128645,v1:172.21.15.5:6809/2565128645] [v2:172.21.15.5:6810/2565128645,v1:172.21.15.5:6811/2565128645] exists,up 02202e5a-e516-4438-9378-7fad80297e64 2024-09-14T07:07:45.272 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/2226792373,v1:172.21.15.110:6801/2226792373] [v2:172.21.15.110:6802/2226792373,v1:172.21.15.110:6803/2226792373] exists,up 57a45940-90af-484d-9570-8de1c6b2252b 2024-09-14T07:07:45.272 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/940255113,v1:172.21.15.110:6817/940255113] [v2:172.21.15.110:6818/940255113,v1:172.21.15.110:6819/940255113] exists,up 8150dd96-bfa0-404f-ae37-f18897bcc93e 2024-09-14T07:07:45.272 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/4012173203,v1:172.21.15.110:6809/4012173203] [v2:172.21.15.110:6810/4012173203,v1:172.21.15.110:6811/4012173203] exists,up 59d63d5d-8ef9-43b2-8d7f-49e8e8385435 2024-09-14T07:07:45.288 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6816/1673964337,v1:172.21.15.5:6817/1673964337] [v2:172.21.15.5:6818/1673964337,v1:172.21.15.5:6819/1673964337] exists,up 9c97eae7-11c5-4962-a62d-04d698e50114', 'osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6800/1704459401,v1:172.21.15.5:6801/1704459401] [v2:172.21.15.5:6802/1704459401,v1:172.21.15.5:6803/1704459401] exists,up 91d33ac7-8429-4237-bb93-1259b318642a', 'osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6808/2565128645,v1:172.21.15.5:6809/2565128645] [v2:172.21.15.5:6810/2565128645,v1:172.21.15.5:6811/2565128645] exists,up 02202e5a-e516-4438-9378-7fad80297e64', 'osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/2226792373,v1:172.21.15.110:6801/2226792373] [v2:172.21.15.110:6802/2226792373,v1:172.21.15.110:6803/2226792373] exists,up 57a45940-90af-484d-9570-8de1c6b2252b', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/940255113,v1:172.21.15.110:6817/940255113] [v2:172.21.15.110:6818/940255113,v1:172.21.15.110:6819/940255113] exists,up 8150dd96-bfa0-404f-ae37-f18897bcc93e', 'osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/4012173203,v1:172.21.15.110:6809/4012173203] [v2:172.21.15.110:6810/4012173203,v1:172.21.15.110:6811/4012173203] exists,up 59d63d5d-8ef9-43b2-8d7f-49e8e8385435'] 2024-09-14T07:07:45.288 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mon.a", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster": "ceph", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "admin_socket_mode": "", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_client_required": "cephx, none", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_cluster_required": "cephx", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_debug": "true", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T07:07:45.406 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_service_required": "cephx", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_supported": "cephx", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio": "true", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio_reap_max": "16", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_async_discard_threads": "0", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_block_size": "4096", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_aio": "false", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T07:07:45.407 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_enable_discard": "false", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_flock_retry": "3", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_inject_crash": "0", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_ioring": "false", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_ioring_hipri": "false", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T07:07:45.408 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_type": "", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_allocator": "hybrid", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_buffered_io": "true", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T07:07:45.409 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_replay_recovery": "false", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_sync_write": "false", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_allocator": "hybrid", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_async_db_compaction": "true", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T07:07:45.410 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_create": "true", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_db_create": "false", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_db_path": "", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_db_size": "0", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_path": "", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T07:07:45.411 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_size": "107374182400", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_wal_create": "false", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_wal_path": "", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs": "true", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T07:07:45.412 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_autotune": "true", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_size": "0", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_type": "2q", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_clone_cow": "true", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T07:07:45.413 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_mode": "none", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_freelist": "false", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T07:07:45.414 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T07:07:45.415 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fail_eio": "false", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T07:07:45.416 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T07:07:45.417 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_blob_size": "0", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T07:07:45.418 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_mem": "512", 2024-09-14T07:07:45.419 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_tracing": "false", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T07:07:45.420 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_write_v2": "false", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_write_v2_random": "false", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T07:07:45.421 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_cluster_require_version": "2", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_require_signatures": "false", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_require_version": "2", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_service_require_signatures": "false", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_service_require_version": "2", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_sign_messages": "true", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "chdir": "", 2024-09-14T07:07:45.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client_acl_type": "", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_asio_thread_count": "2", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_cache_mid": "0.750000", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_cache_size": "16384", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_caps_release_delay": "5", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_check_pool_perm": "true", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_debug_force_sync_read": "false", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_debug_getattr_caps": "false", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_debug_inject_features": "", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_die_on_failed_remount": "false", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_dirsize_rbytes": "true", 2024-09-14T07:07:45.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_force_lazyio": "false", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_fs": "", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_inject_release_failure": "false", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_max_inline_size": "4096", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mds_namespace": "", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_metadata": "", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mount_gid": "-1", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mount_timeout": "300", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mount_uid": "-1", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mountpoint": "/", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_notify_timeout": "10", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc": "true", 2024-09-14T07:07:45.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_max_objects": "1000", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_size": "209715200", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_permissions": "true", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_quota": "true", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_quota_df": "true", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_readahead_max_bytes": "0", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_readahead_max_periods": "4", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_readahead_min": "131072", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_reconnect_stale": "false", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_shutdown_timeout": "30", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_snapdir": ".snap", 2024-09-14T07:07:45.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_tick_interval": "1", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "client_trace": "", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "client_use_faked_inos": "false", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "client_use_random_mds": "false", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_graylog": "false", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_graylog_port": "12201", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_monitors": "default=true", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_syslog": "false", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_syslog_level": "info", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "-", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_network": "", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_network_interface": "", 2024-09-14T07:07:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zlib_isal": "false", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zlib_level": "5", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zstd_level": "1", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_location": "", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_location_hook": "", 2024-09-14T07:07:45.427 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_location_hook_timeout": "10", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "daemonize": "false", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "daos_pool": "tank", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_alienstore": "0/5", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_asok": "1/5", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_asok_assert_abort": "false", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_auth": "20/20", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_bdev": "1/3", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_bluefs": "1/5", 2024-09-14T07:07:45.428 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_bluestore": "1/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_buffer": "0/1", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_ceph_dedup": "0/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_cephsqlite": "0/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_client": "0/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_compressor": "1/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_context": "0/1", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_crush": "1/1", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_crypto": "1/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_cyanstore": "0/5", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T07:07:45.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_dpdk": "1/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_eventtrace": "1/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_filer": "0/1", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_filestore": "1/3", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_finisher": "1/1", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_fuse": "1/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_javaclient": "1/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_journal": "1/3", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_journaler": "0/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_kstore": "1/5", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_lockdep": "0/1", 2024-09-14T07:07:45.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mclock": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_balancer": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_locker": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_log": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_migrator": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_memstore": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mgr": "2/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mgrc": "1/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mon": "20/20", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_monc": "0/10", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_ms": "1/1", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_none": "0/5", 2024-09-14T07:07:45.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_objclass": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_objectcacher": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_objecter": "0/1", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_optracker": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_osd": "1/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_paxos": "20/20", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_perfcounter": "1/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_prioritycache": "1/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rados": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd_replay": "0/5", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_refs": "0/0", 2024-09-14T07:07:45.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_reserver": "1/1", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_access": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_flight": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_notification": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_sync": "1/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rocksdb": "4/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore": "0/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_backref": "0/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_cache": "0/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T07:07:45.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_device": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_epm": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_journal": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_lba": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_odata": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_omap": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_onode": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_t": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_tm": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_striper": "0/1", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_test": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_throttle": "1/1", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_timer": "0/1", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_tp": "0/5", 2024-09-14T07:07:45.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_trace": "1/5", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "device_failure_prediction_mode": "none", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_graylog": "false", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_journald": "false", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_stderr": "true", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_syslog": "false", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event_tracing": "false", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_cert_file": "", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_http_port": "9926", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_key_file": "", 2024-09-14T07:07:45.435 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_prio_limit": "5", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_sort_metrics": "true", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_stats_period": "5", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "fake_statfs_for_testing": "0", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "fatal_signal_handlers": "true", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filer_max_purge_ops": "10", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filer_max_truncate_ops": "128", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_blackhole": "false", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_btrfs_snap": "true", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_caller_concurrency": "10", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T07:07:45.436 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_debug_omap_check": "false", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_debug_verify_split": "false", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_dump_file": "", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fadvise": "true", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fail_eio": "true", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fd_cache_size": "128", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fiemap": "false", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T07:07:45.437 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_inject_stall": "0", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_journal_parallel": "false", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_journal_trailing": "false", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_journal_writeahead": "false", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_kill_at": "0", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T07:07:45.438 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_merge_threshold": "-10", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_odsync_write": "false", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_omap_backend_path": "", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_op_threads": "2", 2024-09-14T07:07:45.439 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_punch_hole": "false", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_ops": "50", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_seek_data_hole": "false", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_sloppy_crc": "false", 2024-09-14T07:07:45.440 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_splice": "false", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_split_multiple": "2", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_split_rand_factor": "20", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_update_to": "1000", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T07:07:45.441 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_xfs_extsize": "false", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_zfs_snap": "false", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fio_dir": "/tmp/fio", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_allow_other": "true", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_big_writes": "true", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_debug": "false", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_default_permissions": "false", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_disable_pagecache": "false", 2024-09-14T07:07:45.442 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_max_write": "0", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_multithreaded": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_require_active_mds": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_set_user_groups": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_splice_move": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_splice_read": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_splice_write": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "gss_target_name": "ceph", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_file": "", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_inject_failure": "0", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_interval": "5", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "host": "smithi005", 2024-09-14T07:07:45.443 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "inject_early_sigterm": "false", 2024-09-14T07:07:45.444 INFO:teuthology.orchestra.run.smithi005.stdout: "jaeger_agent_port": "6799", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "jaeger_tracing_enable": "false", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_aio": "true", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_align_min_size": "65536", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_block_align": "true", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_block_size": "4096", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_dio": "true", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_discard": "false", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_force_aio": "false", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_ignore_corruption": "false", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_max_write_entries": "100", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_replay_from": "0", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T07:07:45.445 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_write_header_frequency": "0", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_zero_on_create": "false", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "journaler_prefetch_periods": "10", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "journaler_prezero_periods": "5", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "journaler_write_head_interval": "15", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "keyfile": "", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_backend": "rocksdb", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T07:07:45.446 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_max_bytes": "67108864", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_max_ops": "512", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_onode_map_size": "1024", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_sync_transaction": "false", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "librados_thread_count": "2", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "lockdep": "false", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "lockdep_force_backtrace": "false", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_coarse_timestamps": "true", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_flush_on_exit": "false", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T07:07:45.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_graylog_port": "12201", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_max_new": "1000", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_max_recent": "10000", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_stderr_prefix": "", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": "true", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_graylog": "false", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_journald": "false", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_stderr": "false", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_syslog": "false", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_action_on_write_error": "1", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_alternate_name_max": "8192", 2024-09-14T07:07:45.448 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_asio_thread_count": "2", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_export_pin": "true", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_interval": "10", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_max": "-1", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_max_until": "-1", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_merge_size": "50", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T07:07:45.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_mode": "0", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_bits": "3", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_size": "10000", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T07:07:45.450 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_mid": "0.700000", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T07:07:45.451 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_trim_interval": "1", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_auth_pins": "false", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_frag": "false", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_scatterstat": "false", 2024-09-14T07:07:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_subtrees": "false", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_default_dir_hash": "2", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_defer_session_stale": "true", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_max_entries": "0", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_prefetch": "true", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T07:07:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_early_reply": "true", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_enable_op_tracker": "true", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_enforce_unique_name": "true", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T07:07:45.454 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_health_dummy": "false", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_join_fs": "", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_journal_format": "1", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_create_at": "0", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_dirfrag_at": "0", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_export_at": "0", 2024-09-14T07:07:45.455 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_import_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_journal_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_link_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_openc_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_rename_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T07:07:45.456 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_max_events": "-1", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_max_segments": "128", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_pause": "false", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_segment_size": "0", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_trim_threshold": "128", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_completed_requests": "100000", 2024-09-14T07:07:45.457 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_export_size": "20971520", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_file_recover": "32", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_purge_files": "64", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_purge_ops": "8192", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_metrics_update_interval": "2", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_min_caps_per_client": "100", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_numa_node": "-1", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T07:07:45.458 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_duration": "600", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_size": "20", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_log_threshold": "5", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_ping_grace": "15", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_ping_interval": "5", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_max_caps": "30000", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T07:07:45.459 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_replay_interval": "1.000000", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_root_ino_gid": "0", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_root_ino_uid": "0", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_scrub_stats_review_period": "1", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T07:07:45.460 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_shutdown_check": "0", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_skip_ino": "0", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_snap_min_uid": "0", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_snap_rstat": "false", 2024-09-14T07:07:45.461 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_symlink_recovery": "true", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_thrash_exports": "0", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_thrash_fragments": "0", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_tick_interval": "5.000000", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_valgrind_exit": "false", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_verify_backtrace": "1", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_verify_scatter": "false", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_wipe_sessions": "false", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool_debug": "false", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T07:07:45.462 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_page_set": "false", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_page_size": "65536", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_client_bytes": "134217728", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_client_messages": "512", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_disabled_modules": "", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_max_pg_creating": "1024", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mds_messages": "128", 2024-09-14T07:07:45.463 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mon_messages": "128", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_op_history_duration": "600", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_op_history_size": "20", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_op_log_threshold": "5", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_osd_messages": "8192", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_pool": "true", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T07:07:45.464 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_standby_modules": "true", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_stats_period": "5", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_stats_threshold": "5", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_tick_period": "2", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_allow_pool_delete": "true", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_bytes": "104857600", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T07:07:45.465 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_target_rank": "-1", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_level": "debug", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T07:07:45.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_compact_on_start": "false", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_compact_on_trim": "true", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cpu_threads": "4", 2024-09-14T07:07:45.467 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data_avail_crit": "5", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data_avail_warn": "5", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_dump_json": "false", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_extra_checks": "false", 2024-09-14T07:07:45.468 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_down_uptime_grace": "60", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_election_default_strategy": "1", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_election_timeout": "5.000000", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T07:07:45.469 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_enable_op_tracker": "true", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_fake_pool_delete": "false", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_force_quorum_join": "false", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_log_update_period": "5", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_max_detail": "50", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_to_clog": "true", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_host": "172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.110", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_host_override": "", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_initial_members": "", 2024-09-14T07:07:45.470 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_lease": "5.000000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_log_full_interval": "50", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_log_max": "10000", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_log_max_summary": "50", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_log_epochs": "500", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T07:07:45.471 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_osd": "10000", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_memory_autotune": "true", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_memory_target": "2147483648", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_digest_period": "5", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T07:07:45.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_complaint_time": "30", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_duration": "600", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_size": "20", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_log_threshold": "5", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T07:07:45.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_cache_size": "500", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T07:07:45.474 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_report_timeout": "900", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T07:07:45.475 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T07:07:45.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_max_osds": "4", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_interval": "86400", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_max_keys": "100", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_timeout": "300", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_session_timeout": "300", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_smart_report_timeout": "5", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T07:07:45.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_pool_size": "4", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_debug": "false", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_tick_interval": "5", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T07:07:45.478 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T07:07:45.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_older_version": "true", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "monmap": "", 2024-09-14T07:07:45.480 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_tracing_enabled": "false", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_op_threads": "3", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_cm": "false", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_device_name": "", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T07:07:45.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_sl": "3", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_type": "ib", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_reap_threshold": "5", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_before_connect": "false", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_ipv4": "true", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_ipv6": "false", 2024-09-14T07:07:45.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_msgr1": "true", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_msgr2": "true", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_port_max": "7568", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_port_min": "6800", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_retry_count": "3", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_retry_delay": "5", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_client": "false", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_mds": "false", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_mgr": "false", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_mon": "false", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_osd": "false", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_client_mode": "crc secure", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-14T07:07:45.483 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_cluster_type": "", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_compress_secure": "false", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_crc_data": "true", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_crc_header": "true", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_bug": "true", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_old_message": "true", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T07:07:45.484 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_hugepages": "", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_lro": "true", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_pmd": "", 2024-09-14T07:07:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_port_id": "0", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dump_on_send": "false", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_delay_type": "", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_network_congestion": "0", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_max_accept_failures": "4", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_max_backoff": "15.000000", 2024-09-14T07:07:45.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_osd_compress_mode": "none", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_pq_min_cost": "65536", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_public_type": "", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_service_mode": "crc secure", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_nodelay": "true", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T07:07:45.487 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_type": "async+posix", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "no_config_file": "false", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_inflight_ops": "1024", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_timeout": "10.000000", 2024-09-14T07:07:45.488 INFO:teuthology.orchestra.run.smithi005.stdout: "objectstore_blackhole": "false", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "openssl_engine_opts": "", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_max_ops": "4", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_slop": "0.020000", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_asio_thread_count": "2", 2024-09-14T07:07:45.489 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backfill_scan_max": "512", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backfill_scan_min": "64", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backoff_on_peering": "false", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_beacon_report_interval": "300", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_duration": "30", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_blkin_trace_all": "false", 2024-09-14T07:07:45.490 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_update_on_start": "true", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_message_cap": "256", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_op_priority": "63", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_watch_timeout": "30", 2024-09-14T07:07:45.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_command_max_records": "256", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_compact_on_start": "false", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_update_on_start": "true", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T07:07:45.492 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_op_order": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_shutdown": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T07:07:45.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_trim_objects": "false", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_default_notify_timeout": "30", 2024-09-14T07:07:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_ec_partial_reads": "true", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_enable_op_tracker": "true", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_info": "true", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_shutdown": "true", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T07:07:45.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_function_tracing": "false", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_grace": "20", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_interval": "6", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_stale": "600", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T07:07:45.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_journal_size": "5120", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_kill_backfill_at": "0", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_cache_size": "50", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_dedup": "true", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_message_max": "40", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T07:07:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_attr_name_len": "100", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_attr_size": "0", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_backfills": "1", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_markdown_count": "5", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_markdown_period": "600", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_object_name_len": "2048", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_object_size": "134217728", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pgls": "1024", 2024-09-14T07:07:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_push_cost": "8388608", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_push_objects": "10", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_scrubs": "3", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_write_size": "90", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T07:07:45.499 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_profile": "balanced", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_base": "805306368", 2024-09-14T07:07:45.500 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_target": "4294967296", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_target_autotune": "false", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_min_recovery_priority": "0", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_report_interval": "5", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_num_cache_shards": "32", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T07:07:45.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_numa_node": "-1", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objecter_finishers": "1", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objectstore": "bluestore", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objectstore_fuse": "false", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objectstore_tracing": "false", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_duration": "600", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_size": "20", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T07:07:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_log_threshold": "5", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_shards": "0", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_thread_timeout": "15", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_open_classes_on_start": "true", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_os_flags": "0", 2024-09-14T07:07:45.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_peering_op_priority": "255", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_delete_priority": "5", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T07:07:45.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_crimson": "false", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flags": "0", 2024-09-14T07:07:45.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_min_size": "0", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_size": "2", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_type": "replicated", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T07:07:45.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_cost": "20971520", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_active": "0", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_op_priority": "3", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_priority": "5", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T07:07:45.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_repair_during_recovery": "false", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T07:07:45.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_cost": "52428800", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_end_hour": "0", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_priority": "5", 2024-09-14T07:07:45.509 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_skip_data_digest": "false", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_smart_report_timeout": "5", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_priority": "5", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T07:07:45.510 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_target_transaction_size": "30", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tracing": "false", 2024-09-14T07:07:45.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_use_stale_snap": "false", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_kill_at": "0", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_max_join_drift": "10", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_min": "500", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_min_wait": "0.050000", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_service_trim_max": "500", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_service_trim_min": "250", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_stash_full_interval": "25", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_trim_max": "500", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_trim_min": "250", 2024-09-14T07:07:45.512 INFO:teuthology.orchestra.run.smithi005.stdout: "perf": "true", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "-", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrv": "", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "public_bind_addr": "-", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "public_network": "", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "public_network_interface": "", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "qat_compressor_enabled": "false", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "rados_mon_op_timeout": "0", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "rados_osd_op_timeout": "0", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "rados_tracing": "false", 2024-09-14T07:07:45.513 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_atime_update_interval": "60", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache": "true", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_policy": "writearound", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_size": "33554432", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T07:07:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_compression_hint": "none", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_clone_format": "auto", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_data_pool": "", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_features": "61", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_format": "2", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_map_options": "", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_order": "22", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_pool": "rbd", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_stripe_count": "0", 2024-09-14T07:07:45.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_io_scheduler": "simple", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T07:07:45.516 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_order": "24", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_pool": "", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_splay_width": "4", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T07:07:45.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T07:07:45.518 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_op_threads": "1", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T07:07:45.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_plugins": "", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T07:07:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T07:07:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_tracing": "false", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_validate_names": "true", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_validate_pool": "true", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "restapi_base_url": "", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "restapi_log_level": "", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_admin_entry": "admin", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T07:07:45.522 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_asio_assert_yielding": "false", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_backend_store": "rados", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_barbican_url": "", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_beast_enable_async": "true", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cache_enabled": "true", 2024-09-14T07:07:45.523 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_config_store": "rados", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_content_length_compat": "false", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cross_domain_policy": "", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T07:07:45.524 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T07:07:45.525 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T07:07:45.526 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T07:07:45.527 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_window": "30", 2024-09-14T07:07:45.528 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_disable_s3select": "false", 2024-09-14T07:07:45.529 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dns_name": "", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dns_s3website_name": "", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T07:07:45.530 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_mdsearch": "true", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_ops_log": "false", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_static_website": "false", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_usage_log": "false", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_expose_bucket": "false", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_extended_http_attrs": "", 2024-09-14T07:07:45.531 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_filter": "none", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_objs": "32", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T07:07:45.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_init_timeout": "300", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_password": "", 2024-09-14T07:07:45.533 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_project": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_token": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_user": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T07:07:45.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_url": "", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_objs": "32", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_worker": "3", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T07:07:45.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_uri": "", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_http_headers": "", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T07:07:45.536 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_attr_size": "0", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_listing_results": "1000", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_notify_retries": "10", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T07:07:45.537 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T07:07:45.538 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_num_control_oids": "8", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_numa_node": "-1", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T07:07:45.539 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_op_tracing": "false", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_opa_token": "", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_opa_url": "", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_rados": "false", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T07:07:45.540 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_print_continue": "true", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T07:07:45.541 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rados_tracing": "false", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_realm": "", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_realm_id": "", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_region": "", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_request_uri": "", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T07:07:45.542 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_resolve_cname": "false", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_run_sync_thread": "true", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T07:07:45.543 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_script_uri": "", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_service_provider_name": "", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_client_id": "", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_client_secret": "", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_entry": "sts", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_key": "sts", 2024-09-14T07:07:45.544 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_auth_url": "", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_custom_header": "", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_need_stats": "true", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_tenant_name": "", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_url": "", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T07:07:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_lease_period": "120", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_thread_pool_size": "512", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T07:07:45.546 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_comment": "", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_createby": "", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_encoding": "", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_flag": "false", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_origin": "", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_tracker": "", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_max_shards": "32", 2024-09-14T07:07:45.547 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_use_opa_authz": "false", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_counters_cache": "false", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_unique_email": "true", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_verify_ssl": "true", 2024-09-14T07:07:45.548 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zone": "", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zone_id": "", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zonegroup": "", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zonegroup_id": "", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_block_size": "4096", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T07:07:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_partition_filters": "false", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_perf": "false", 2024-09-14T07:07:45.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "run_dir": "/var/run/ceph", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_block_create": "true", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T07:07:45.551 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_device_size": "53687091200", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_full_integrity_check": "false", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_main_device_type": "SSD", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T07:07:45.552 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_segment_size": "67108864", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "set_keepcaps": "false", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "setgroup": "", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "setuser": "", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "setuser_match_path": "", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "thp": "false", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "threadpool_default_timeout": "60", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "throttler_perf_counter": "true", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tmp_dir": "/tmp", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "uadk_compressor_enabled": "false", 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-14T07:07:45.553 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:45.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T07:07:45.695 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a: {} 2024-09-14T07:07:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:07:45.708 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b: {} 2024-09-14T07:07:45.708 INFO:teuthology.orchestra.run.smithi005.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:07:45.718 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c: {} 2024-09-14T07:07:45.718 INFO:teuthology.orchestra.run.smithi005.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:07:45.733 DEBUG:teuthology.orchestra.run.smithi005:> type ceph-objectstore-tool 2024-09-14T07:07:45.737 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T07:07:45.737 DEBUG:teuthology.orchestra.run.smithi110:> type ceph-objectstore-tool 2024-09-14T07:07:45.743 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T07:07:45.743 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T07:07:45.753 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T07:07:45.753 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T07:07:45.753 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:07:45.753 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:07:45.753 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-14T07:07:45.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-14T07:07:45.754 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T07:07:45.754 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T07:07:45.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:07:45.755 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T07:07:45.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:07:45.755 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T07:07:45.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:07:45.756 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-14T07:07:45.788 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:45.786+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:45.805 INFO:teuthology.orchestra.run.smithi145.stdout:mapped writes DISABLED 2024-09-14T07:07:45.805 INFO:teuthology.orchestra.run.smithi145.stdout:mapped reads DISABLED 2024-09-14T07:07:45.806 INFO:teuthology.orchestra.run.smithi145.stdout:Seed set to 9637 2024-09-14T07:07:45.806 INFO:teuthology.orchestra.run.smithi145.stdout:rbd-nbd mode enabled 2024-09-14T07:07:45.856 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:45.855+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:45.894 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:45.894 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:07:45.894 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:45.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:07:45.911 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:07:45.912 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:07:45.928 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:07:45.929 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:07:45.944 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:07:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:07:45.957 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:45.954+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:45.965 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:07:45.965 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:07:45.985 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:07:45.986 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:07:46.006 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:07:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:07:46.022 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:46.022 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:46.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:07:46.058 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:46.054+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:46.119 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:07:46.119+0000 7f1e6b537640 -1 mon.a@0(leader).osd e20 definitely_dead 0 2024-09-14T07:07:46.159 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:46.159+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:46.191 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:46.192 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:46.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:07:46.291 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:46.291+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:46.362 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:46.359+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:46.366 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:46.366 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:46.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:07:46.463 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:46.462+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:46.545 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:46.545 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:46.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:07:46.564 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:46.562+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:46.665 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:46.663+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:46.677 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:46.677 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:46.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:07:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.2. 2024-09-14T07:07:46.765 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:46.762+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:07:46.866 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:46.863+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:46.874 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:46.875 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:46.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:07:47.072 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:47.071+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:47.174 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:47.171+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:47.202 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:47.202 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:47.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:07:47.274 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:47.270+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:47.375 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:47.374+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:47.385 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:47.385 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:47.398 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:07:47.476 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:47.475+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:47.530 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:47.531 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:47.545 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:07:47.577 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:47.575+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:47.661 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:47.661 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:47.677 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:07:47.678 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:47.679+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:47.780 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:47.774+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:47.817 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:47.817 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:47.831 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:07:47.879 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:47.879+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:47.980 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:47.979+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:47.997 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:47.997 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:48.011 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:07:48.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:07:48.081 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:48.078+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:48.134 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:48.134 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:48.148 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:07:48.178 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:07:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:07:48.191 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:48.190+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:48.195 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:07:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:07:48.212 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:07:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:07:48.232 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:07:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:07:48.254 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:07:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:07:48.277 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:07:48.277 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:07:48.283 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:48.283+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:48.316 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:07:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:07:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [] 2024-09-14T07:07:48.316 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:48.330 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:07:48.384 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:48.382+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:48.450 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:48.450 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:48.450 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:07:48.450 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:48.463 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:07:48.508 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:48.506+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:48.585 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:48.582+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:48.632 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:07:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:07:48.632 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:48.646 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:07:48.687 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:48.686+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:48.787 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:48.787+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:48.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:07:48.812 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:48.812 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4199.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:36.022070+0000", 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.789686527000001, 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021902689999999999, 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4199", 2024-09-14T07:07:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.5:0/4051694687", 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022070+0000", 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022070+0000", 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022078+0000", 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3000000000000002e-06 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022089+0000", 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1070000000000001e-05 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022112+0000", 2024-09-14T07:07:48.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2770000000000001e-05 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022128+0000", 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6160000000000001e-05 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022242+0000", 2024-09-14T07:07:48.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114045 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022476+0000", 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000233952 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.022687+0000", 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000210638 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.023767+0000", 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010805649999999999 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.024097+0000", 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032943399999999998 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.024172+0000", 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5512999999999993e-05 2024-09-14T07:07:48.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.024260+0000", 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7822e-05 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop_reply(client.4199.0:1 2.3 e17/16 ondisk, result = 0)", 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:36.023949+0000", 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.787807415, 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032324099999999999, 2024-09-14T07:07:48.819 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.023949+0000", 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.023949+0000", 2024-09-14T07:07:48.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.023950+0000", 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5600000000000001e-06 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.023959+0000", 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8070000000000006e-06 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.023980+0000", 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0809e-05 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.023994+0000", 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3653e-05 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.024078+0000", 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4149999999999999e-05 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:48.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.024089+0000", 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.116e-05 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.024272+0000", 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018310200000000001 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4199.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:07:48.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:36.028541+0000", 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.783215283000001, 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051792299999999999, 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4199", 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.5:0/4051694687", 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.028541+0000", 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.028541+0000", 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.028543+0000", 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.747e-06 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.028553+0000", 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0132999999999999e-05 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.028577+0000", 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4202999999999998e-05 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.028590+0000", 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3173000000000001e-05 2024-09-14T07:07:48.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.028738+0000", 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014760199999999999 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029059+0000", 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032106499999999999 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4199.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:36.029399+0000", 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.782356892999999, 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033143099999999998, 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4199", 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.5:0/4051694687", 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3 2024-09-14T07:07:48.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029399+0000", 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029399+0000", 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029401+0000", 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.64e-06 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029408+0000", 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2699999999999999e-06 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029427+0000", 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8919999999999998e-05 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029446+0000", 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9383000000000001e-05 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029538+0000", 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1578999999999999e-05 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:36.029731+0000", 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019263900000000001 2024-09-14T07:07:48.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4199.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:37.024310+0000", 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.787446264, 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064646900000000004, 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4199", 2024-09-14T07:07:48.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.5:0/4051694687", 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 5 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024310+0000", 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024310+0000", 2024-09-14T07:07:48.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024313+0000", 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9000000000000002e-06 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024334+0000", 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0743000000000001e-05 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024367+0000", 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3683000000000003e-05 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024385+0000", 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7413000000000001e-05 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024514+0000", 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012901899999999999 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024525+0000", 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0837e-05 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.027793+0000", 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032689339999999998 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.027986+0000", 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000192222 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.028179+0000", 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019309899999999999 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.029590+0000", 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014117229999999999 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030498+0000", 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000907266 2024-09-14T07:07:48.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030649+0000", 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015092600000000001 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030775+0000", 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012592500000000001 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop_reply(client.4199.0:5 2.2 e18/16 ondisk, result = 0)", 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:37.030279+0000", 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.781477089999999, 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051820299999999996, 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030279+0000", 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030279+0000", 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030281+0000", 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9300000000000002e-06 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030291+0000", 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7030000000000001e-06 2024-09-14T07:07:48.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030319+0000", 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8353000000000001e-05 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030334+0000", 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4683e-05 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030453+0000", 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119399 2024-09-14T07:07:48.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030481+0000", 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.811e-05 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030797+0000", 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000316025 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4199.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:38.032884+0000", 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.778872469, 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066585289999999998, 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4199", 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.5:0/4051694687", 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 7 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032884+0000", 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032884+0000", 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032887+0000", 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.444e-06 2024-09-14T07:07:48.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032902+0000", 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4430000000000001e-05 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032937+0000", 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5366e-05 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032963+0000", 2024-09-14T07:07:48.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.654e-05 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.033116+0000", 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015216900000000001 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.033135+0000", 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8960000000000001e-05 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.036494+0000", 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033589869999999999 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.036625+0000", 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131159 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.036837+0000", 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000211759 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.038340+0000", 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001503211 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039391+0000", 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001051202 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039480+0000", 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9069999999999999e-05 2024-09-14T07:07:48.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039542+0000", 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2231999999999996e-05 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop_reply(client.4199.0:7 2.2 e19/16 ondisk, result = 0)", 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:38.039202+0000", 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.772554337000001, 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035427400000000002, 2024-09-14T07:07:48.849 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039202+0000", 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039202+0000", 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039203+0000", 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5239999999999999e-06 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039211+0000", 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7800000000000001e-06 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039235+0000", 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4105999999999999e-05 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039250+0000", 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4493e-05 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039368+0000", 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117806 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039380+0000", 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2262999999999999e-05 2024-09-14T07:07:48.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039556+0000", 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017630200000000001 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:48.853 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:48.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:07:48.899 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:48.899+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:48.979 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:48.979 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:07:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:48.989 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:48.986+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:48.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:07:49.090 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:49.091+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:07:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:07:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:49.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:07:49.191 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:49.190+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:49.293 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:49.291+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:49.342 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:07:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:07:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:07:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:20 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'10, mlcod=14'8)", 2024-09-14T07:07:49.342 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.694878969000001, 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002918571, 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:07:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645365+0000", 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7799999999999999e-06 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645608+0000", 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024235200000000001 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645637+0000", 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9315999999999998e-05 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645651+0000", 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.375e-05 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645777+0000", 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012618199999999999 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645830+0000", 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3340000000000001e-05 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648125+0000", 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022950549999999998 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648282+0000", 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015679600000000001 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:21 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'11, mlcod=14'8)", 2024-09-14T07:07:49.347 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.692291645000001, 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041484010000000003, 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647953+0000", 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.79e-06 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648081+0000", 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000128102 2024-09-14T07:07:49.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648114+0000", 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3269000000000003e-05 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648150+0000", 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5939999999999998e-05 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648364+0000", 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000214415 2024-09-14T07:07:49.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648418+0000", 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3386e-05 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652012+0000", 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035942700000000001 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652099+0000", 2024-09-14T07:07:49.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7229000000000001e-05 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:22 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'12, mlcod=14'8)", 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.690031318999999, 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041167649999999997, 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650213+0000", 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7129999999999999e-06 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650325+0000", 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011184900000000001 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650354+0000", 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9595999999999999e-05 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650385+0000", 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0179999999999999e-05 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650512+0000", 2024-09-14T07:07:49.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012745899999999999 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650567+0000", 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4447000000000002e-05 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654240+0000", 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036733109999999999 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654328+0000", 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8209999999999997e-05 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:23 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'13, mlcod=14'8)", 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.652519+0000", 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.687724029999998, 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023625009999999999, 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.357 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652519+0000", 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652519+0000", 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652520+0000", 2024-09-14T07:07:49.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.733e-06 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652578+0000", 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7679999999999997e-05 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652607+0000", 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8989e-05 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652621+0000", 2024-09-14T07:07:49.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4477e-05 2024-09-14T07:07:49.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652760+0000", 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000138942 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652813+0000", 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2707000000000002e-05 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654784+0000", 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019710140000000001 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654881+0000", 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6959000000000005e-05 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.363 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:24 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'14, mlcod=14'8)", 2024-09-14T07:07:49.363 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.654792+0000", 2024-09-14T07:07:49.363 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.685450937999999, 2024-09-14T07:07:49.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024315309999999998, 2024-09-14T07:07:49.363 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654792+0000", 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654792+0000", 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654793+0000", 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4130000000000001e-06 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654882+0000", 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9293e-05 2024-09-14T07:07:49.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654918+0000", 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5862999999999999e-05 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654935+0000", 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6739999999999999e-05 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.655045+0000", 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110105 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.655077+0000", 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1464000000000002e-05 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.657146+0000", 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002068987 2024-09-14T07:07:49.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.657223+0000", 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7665999999999998e-05 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:54 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'36, mlcod=14'33)", 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.782677+0000", 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.557565832999998, 2024-09-14T07:07:49.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0025255899999999999, 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782677+0000", 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782677+0000", 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782677+0000", 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1099999999999995e-07 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782742+0000", 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4591000000000006e-05 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782760+0000", 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7725999999999999e-05 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782779+0000", 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8915000000000001e-05 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782846+0000", 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7607000000000005e-05 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.782875+0000", 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8461000000000001e-05 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.785140+0000", 2024-09-14T07:07:49.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002265001 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.785202+0000", 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2577999999999996e-05 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:65 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'43, mlcod=14'40)", 2024-09-14T07:07:49.373 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.381412007000002, 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032602360000000001, 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958832+0000", 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0249999999999999e-06 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958839+0000", 2024-09-14T07:07:49.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6979999999999998e-06 2024-09-14T07:07:49.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958857+0000", 2024-09-14T07:07:49.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:07:49.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958865+0000", 2024-09-14T07:07:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:07:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958964+0000", 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8787000000000001e-05 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959008+0000", 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3720000000000002e-05 2024-09-14T07:07:49.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962018+0000", 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030102599999999998 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962091+0000", 2024-09-14T07:07:49.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2985999999999995e-05 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:66 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'44, mlcod=14'40)", 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.380824359999998, 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045205219999999999, 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.380 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959419+0000", 2024-09-14T07:07:49.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959471+0000", 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1078000000000001e-05 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959484+0000", 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3083e-05 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959493+0000", 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5540000000000003e-06 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959682+0000", 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018848700000000001 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959722+0000", 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0784000000000001e-05 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963902+0000", 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041791110000000001 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963939+0000", 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7212000000000002e-05 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:67 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'45, mlcod=14'40)", 2024-09-14T07:07:49.384 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.379670816000001, 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044867969999999998, 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960573+0000", 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8100000000000001e-07 2024-09-14T07:07:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960582+0000", 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960598+0000", 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6184999999999998e-05 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960605+0000", 2024-09-14T07:07:49.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3320000000000003e-06 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961328+0000", 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000723075 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961373+0000", 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4552000000000003e-05 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965024+0000", 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036513520000000001 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965059+0000", 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4430000000000001e-05 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'46, mlcod=14'41)", 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.378926505999999, 2024-09-14T07:07:49.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003788769, 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961317+0000", 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.02e-06 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961326+0000", 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5539999999999998e-06 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961341+0000", 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5e-05 2024-09-14T07:07:49.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961348+0000", 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8310000000000001e-06 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962174+0000", 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082531300000000004 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962220+0000", 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6338000000000003e-05 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965063+0000", 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028425590000000001 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965105+0000", 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2154000000000003e-05 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:69 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'47, mlcod=14'41)", 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.378305632, 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032062459999999998, 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961938+0000", 2024-09-14T07:07:49.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961974+0000", 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6205000000000003e-05 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961993+0000", 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8286000000000001e-05 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962000+0000", 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4279999999999999e-06 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962847+0000", 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084736100000000003 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962865+0000", 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8028000000000001e-05 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965112+0000", 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022461199999999999 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965143+0000", 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1687999999999999e-05 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:71 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'49, mlcod=14'42)", 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.376619813000001, 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036766120000000001, 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:07:49.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963624+0000", 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3900000000000003e-07 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963632+0000", 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8359999999999992e-06 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963650+0000", 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8130000000000001e-05 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963659+0000", 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2779999999999999e-06 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964704+0000", 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001044683 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964720+0000", 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6456e-05 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967251+0000", 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002531307 2024-09-14T07:07:49.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967299+0000", 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7982999999999998e-05 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:72 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'50, mlcod=14'42)", 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.376030709999998, 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046904399999999997, 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964213+0000", 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8400000000000003e-07 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964294+0000", 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0747000000000005e-05 2024-09-14T07:07:49.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964306+0000", 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2155e-05 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964314+0000", 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0090000000000007e-06 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965268+0000", 2024-09-14T07:07:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095471900000000003 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965316+0000", 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7117000000000003e-05 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968866+0000", 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003550311 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968902+0000", 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6498000000000002e-05 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:73 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'51, mlcod=14'42)", 2024-09-14T07:07:49.408 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.374817974999999, 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035339820000000002, 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965441+0000", 2024-09-14T07:07:49.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6181999999999999e-05 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965453+0000", 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1252e-05 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965460+0000", 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-09-14T07:07:49.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966961+0000", 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015005929999999999 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966978+0000", 2024-09-14T07:07:49.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6991999999999999e-05 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968907+0000", 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019290329999999999 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968959+0000", 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2033999999999997e-05 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:74 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'52, mlcod=14'47)", 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.966709+0000", 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.373533896000001, 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022907420000000001, 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.414 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966709+0000", 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966709+0000", 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966709+0000", 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966715+0000", 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1649999999999997e-06 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966726+0000", 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966732+0000", 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1229999999999997e-06 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967365+0000", 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063282900000000001 2024-09-14T07:07:49.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967412+0000", 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7747000000000002e-05 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968965+0000", 2024-09-14T07:07:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001552374 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968999+0000", 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4569000000000001e-05 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:75 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'53, mlcod=14'47)", 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.967249+0000", 2024-09-14T07:07:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.372993982000001, 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0025086940000000001, 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967249+0000", 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967249+0000", 2024-09-14T07:07:49.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967250+0000", 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3600000000000002e-07 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967325+0000", 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5078000000000002e-05 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967335+0000", 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0322999999999999e-05 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967341+0000", 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.443e-06 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968058+0000", 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071636300000000003 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968075+0000", 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7142999999999998e-05 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.969721+0000", 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016465270000000001 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.969757+0000", 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5880999999999997e-05 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:77 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'55, mlcod=14'53)", 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.971194+0000", 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.369048458999998, 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002338162, 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971194+0000", 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971194+0000", 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971195+0000", 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2099999999999995e-07 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971261+0000", 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6099999999999994e-05 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971275+0000", 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4093999999999999e-05 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971286+0000", 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1117e-05 2024-09-14T07:07:49.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971379+0000", 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3190999999999996e-05 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971429+0000", 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9756999999999998e-05 2024-09-14T07:07:49.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973495+0000", 2024-09-14T07:07:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020660320000000002 2024-09-14T07:07:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973532+0000", 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7049999999999999e-05 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:78 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'56, mlcod=14'53)", 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.368294981999998, 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046602329999999997, 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972013+0000", 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.444e-05 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972026+0000", 2024-09-14T07:07:49.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3497e-05 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972033+0000", 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.533e-06 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972762+0000", 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072933500000000001 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972782+0000", 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9417000000000001e-05 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976515+0000", 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037336370000000002 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976608+0000", 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2645999999999994e-05 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.433 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:79 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'57, mlcod=14'53)", 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.367287620999999, 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003768225, 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972956+0000", 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3599999999999999e-06 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973052+0000", 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6132999999999996e-05 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973072+0000", 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9307999999999999e-05 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973089+0000", 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7569000000000001e-05 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973597+0000", 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00050773500000000004 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973646+0000", 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8773000000000002e-05 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976625+0000", 2024-09-14T07:07:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002979607 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976723+0000", 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7739999999999996e-05 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.438 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:80 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'58, mlcod=14'53)", 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.366362425999998, 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030979940000000002, 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973881+0000", 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973963+0000", 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2442999999999997e-05 2024-09-14T07:07:49.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973975+0000", 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1595e-05 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973982+0000", 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7789999999999998e-06 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.975181+0000", 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001199809 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.975228+0000", 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6097999999999998e-05 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976917+0000", 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001689091 2024-09-14T07:07:49.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976978+0000", 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1501999999999995e-05 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.443 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:49.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:07:49.445 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:49.442+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:49.546 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:49.542+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:49.617 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:49.617 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:49.617 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:07:49.617 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:49.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:07:49.647 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:49.646+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:49.748 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:49.746+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:49.758 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:49.758 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:49.758 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:07:49.758 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:07:49.759 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:49.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:07:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:07:49.867 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:49.867+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:49.897 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:07:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4199.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:37.029151+0000", 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.869652641, 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015716370000000001, 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029151+0000", 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029151+0000", 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029152+0000", 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029162+0000", 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3000000000000007e-06 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029189+0000", 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7653000000000001e-05 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029215+0000", 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6177e-05 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029328+0000", 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112515 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029359+0000", 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1103999999999999e-05 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.030621+0000", 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001262186 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.030723+0000", 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101809 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4199.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-14T07:07:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.860882399999999, 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017608540000000001, 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-07 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037940+0000", 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8009999999999999e-05 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037993+0000", 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2945999999999998e-05 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.038014+0000", 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0509999999999998e-05 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.038166+0000", 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015273899999999999 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.038234+0000", 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7303000000000003e-05 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.039599+0000", 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013654590000000001 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.039682+0000", 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3250000000000004e-05 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:7 3.1a e23/21 3:5cea7035:::rbd_info:head v 23'1, mlcod=0'0)", 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.100000063, 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021634900000000001, 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798805+0000", 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1999999999999999e-07 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798819+0000", 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798879+0000", 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9373000000000002e-05 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798902+0000", 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2836e-05 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.799063+0000", 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016089899999999999 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.799100+0000", 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7740000000000001e-05 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.800872+0000", 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017718390000000001 2024-09-14T07:07:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.800967+0000", 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5292999999999997e-05 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:49.911 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:49.912 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:07:49.950 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:49.946+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:50.051 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:50.051+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:50.081 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:07:50.081 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:50.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:07:50.183 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:50.183+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:50.254 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:07:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:07:50.255 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:50.255 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:50.250+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:50.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:07:50.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:07:50.355 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:50.354+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:50.432 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4199.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.409958212999999, 2024-09-14T07:07:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00086957099999999997, 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6800000000000001e-07 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023573+0000", 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2889999999999998e-06 2024-09-14T07:07:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023589+0000", 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5548000000000001e-05 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023600+0000", 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0548e-05 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023677+0000", 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7967999999999996e-05 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023707+0000", 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9475000000000002e-05 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.024381+0000", 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00067376100000000004 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.024434+0000", 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3613999999999999e-05 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:50.439 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:50.448 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:07:50.460 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:07:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:07:50.477 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:07:50.477 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:07:50.494 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:07:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:07:50.495 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:50.494+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:50.512 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:07:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:07:50.533 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:07:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:07:50.554 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:07:50.554 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:07:50.557 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:50.554+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:50.582 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:50.582 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:50.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:07:50.657 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:50.655+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:50.758 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:50.754+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:50.761 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:50.761 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:50.775 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:07:50.863 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:50.862+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:50.945 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:50.945 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:50.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:07:50.960 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:50.958+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:51.061 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:51.058+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:51.121 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:51.121 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:51.135 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:07:51.180 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:51.178+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:51.262 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:51.262+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:51.302 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:51.302 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:51.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:07:51.363 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:51.362+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:51.433 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:51.433 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:51.447 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:07:51.465 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:51.462+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:51.566 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:51.562+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:51.616 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:51.616 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:51.630 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:07:51.675 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:51.674+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:51.755 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:51.755 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:51.768 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:51.766+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:51.770 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:07:51.798 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:07:51.798 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:07:51.798 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-14T07:07:51.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-14T07:07:51.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:07:51.864 INFO:teuthology.orchestra.run.smithi145.stdout:1 write 0x2104000 thru 0x2106901 (0x2902 bytes) 2024-09-14T07:07:51.865 INFO:teuthology.orchestra.run.smithi145.stdout:3 read 0x9f4a00 thru 0xa00621 (0xbc22 bytes) 2024-09-14T07:07:51.865 INFO:teuthology.orchestra.run.smithi145.stdout:8 read 0xffe600 thru 0x100d261 (0xec62 bytes) 2024-09-14T07:07:51.868 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:51.866+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:51.894 INFO:teuthology.orchestra.run.smithi145.stdout:10 write 0xb3f7600 thru 0xb404bce (0xd5cf bytes) 2024-09-14T07:07:51.895 INFO:teuthology.orchestra.run.smithi145.stdout:truncating to largest ever: 0xe839e00 2024-09-14T07:07:51.900 INFO:teuthology.orchestra.run.smithi145.stdout:12 trunc from 0xb404c00 to 0xe839e00 2024-09-14T07:07:51.920 INFO:teuthology.orchestra.run.smithi145.stdout:14 write 0xfed600 thru 0xfef0b0 (0x1ab1 bytes) 2024-09-14T07:07:51.921 INFO:teuthology.orchestra.run.smithi145.stdout:15 write 0x66f800 thru 0x674cc4 (0x54c5 bytes) 2024-09-14T07:07:51.921 INFO:teuthology.orchestra.run.smithi145.stdout:17 read 0x870000 thru 0x87ccbb (0xccbc bytes) 2024-09-14T07:07:51.922 INFO:teuthology.orchestra.run.smithi145.stdout:22 write 0x621c800 thru 0x622019f (0x39a0 bytes) 2024-09-14T07:07:51.922 INFO:teuthology.orchestra.run.smithi145.stdout:23 punch from 0x7466a00 to 0x746ba00, (0x5000 bytes) 2024-09-14T07:07:51.953 INFO:teuthology.orchestra.run.smithi145.stdout:25 write 0xe840e00 thru 0xe84b7fc (0xa9fd bytes) 2024-09-14T07:07:51.955 INFO:teuthology.orchestra.run.smithi145.stdout:truncating to largest ever: 0xe8cfe00 2024-09-14T07:07:51.956 INFO:teuthology.orchestra.run.smithi145.stdout:28 trunc from 0xe84b800 to 0xe8cfe00 2024-09-14T07:07:51.956 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:51.956 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:51.969 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:51.966+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:51.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:07:51.978 INFO:teuthology.orchestra.run.smithi145.stdout:29 read 0xbc7c200 thru 0xbc8ab56 (0xe957 bytes) 2024-09-14T07:07:51.979 INFO:teuthology.orchestra.run.smithi145.stdout:31 read 0x6b0a800 thru 0x6b15e6a (0xb66b bytes) 2024-09-14T07:07:51.994 INFO:teuthology.orchestra.run.smithi145.stdout:32 trunc from 0xe8cfe00 to 0x7922600 2024-09-14T07:07:52.016 INFO:teuthology.orchestra.run.smithi145.stdout:33 write 0x1f4c000 thru 0x1f55aff (0x9b00 bytes) 2024-09-14T07:07:52.023 INFO:teuthology.orchestra.run.smithi145.stdout:34 trunc from 0x7922600 to 0x409dc00 2024-09-14T07:07:52.070 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:52.066+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:52.083 INFO:teuthology.orchestra.run.smithi145.stdout:35 write 0x54ff800 thru 0x5504f03 (0x5704 bytes) 2024-09-14T07:07:52.084 INFO:teuthology.orchestra.run.smithi145.stdout:39 read 0x3e4fc00 thru 0x3e5cf11 (0xd312 bytes) 2024-09-14T07:07:52.084 INFO:teuthology.orchestra.run.smithi145.stdout:40 punch from 0x4db9800 to 0x4dc2e00, (0x9600 bytes) 2024-09-14T07:07:52.085 INFO:teuthology.orchestra.run.smithi145.stdout:42 read 0x54d5600 thru 0x54dbfb0 (0x69b1 bytes) 2024-09-14T07:07:52.108 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:52.108 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:52.121 INFO:teuthology.orchestra.run.smithi145.stdout:44 write 0xc021800 thru 0xc0221d1 (0x9d2 bytes) 2024-09-14T07:07:52.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:07:52.125 INFO:teuthology.orchestra.run.smithi145.stdout:45 write 0x5ab8800 thru 0x5abab1b (0x231c bytes) 2024-09-14T07:07:52.125 INFO:teuthology.orchestra.run.smithi145.stdout:46 read 0x6fc2200 thru 0x6fc4adb (0x28dc bytes) 2024-09-14T07:07:52.125 INFO:teuthology.orchestra.run.smithi145.stdout:49 read 0x91a3800 thru 0x91b1067 (0xd868 bytes) 2024-09-14T07:07:52.136 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:07:52.134+0000 7f1e6b537640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-09-14T07:07:52.142 INFO:teuthology.orchestra.run.smithi145.stdout:50 trunc from 0xc022200 to 0x4246a00 2024-09-14T07:07:52.171 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:52.170+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:52.220 INFO:teuthology.orchestra.run.smithi145.stdout:52 write 0x61f6200 thru 0x61ff35a (0x915b bytes) 2024-09-14T07:07:52.251 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:52.252 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:52.259 INFO:teuthology.orchestra.run.smithi145.stdout:55 write 0xadfe600 thru 0xae0e1b1 (0xfbb2 bytes) 2024-09-14T07:07:52.261 INFO:teuthology.orchestra.run.smithi145.stdout:56 punch from 0xa30a000 to 0xa317000, (0xd000 bytes) 2024-09-14T07:07:52.262 INFO:teuthology.orchestra.run.smithi145.stdout:58 read 0x3bd6a00 thru 0x3be4293 (0xd894 bytes) 2024-09-14T07:07:52.262 INFO:teuthology.orchestra.run.smithi145.stdout:60 write 0xab93400 thru 0xaba1a64 (0xe665 bytes) 2024-09-14T07:07:52.262 INFO:teuthology.orchestra.run.smithi145.stdout:64 punch from 0x8c94800 to 0x8c9ee00, (0xa600 bytes) 2024-09-14T07:07:52.265 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:07:52.268 INFO:teuthology.orchestra.run.smithi145.stdout:66 trunc from 0xae0e200 to 0x718f000 2024-09-14T07:07:52.272 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:52.270+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:52.316 INFO:teuthology.orchestra.run.smithi145.stdout:69 punch from 0x57af000 to 0x57b6c00, (0x7c00 bytes) 2024-09-14T07:07:52.318 INFO:teuthology.orchestra.run.smithi145.stdout:71 write 0x1289a00 thru 0x1297e45 (0xe446 bytes) 2024-09-14T07:07:52.318 INFO:teuthology.orchestra.run.smithi145.stdout:72 write 0x2de7c00 thru 0x2df1c7d (0xa07e bytes) 2024-09-14T07:07:52.318 INFO:teuthology.orchestra.run.smithi145.stdout:73 write 0x60bc400 thru 0x60c2ed3 (0x6ad4 bytes) 2024-09-14T07:07:52.320 INFO:teuthology.orchestra.run.smithi145.stdout:74 read 0x5aefe00 thru 0x5af8f93 (0x9194 bytes) 2024-09-14T07:07:52.361 INFO:teuthology.orchestra.run.smithi145.stdout:79 write 0xbbf7200 thru 0xbc02bcd (0xb9ce bytes) 2024-09-14T07:07:52.363 INFO:teuthology.orchestra.run.smithi145.stdout:80 punch from 0x7fa0000 to 0x7faf600, (0xf600 bytes) 2024-09-14T07:07:52.363 INFO:teuthology.orchestra.run.smithi145.stdout:82 punch from 0x6079e00 to 0x607b400, (0x1600 bytes) 2024-09-14T07:07:52.373 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:52.370+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:52.396 INFO:teuthology.orchestra.run.smithi145.stdout:83 write 0xed26c00 thru 0xed2973e (0x2b3f bytes) 2024-09-14T07:07:52.399 INFO:teuthology.orchestra.run.smithi145.stdout:84 read 0x9c81a00 thru 0x9c84118 (0x2719 bytes) 2024-09-14T07:07:52.399 INFO:teuthology.orchestra.run.smithi145.stdout:85 write 0x6aece00 thru 0x6af40ba (0x72bb bytes) 2024-09-14T07:07:52.399 INFO:teuthology.orchestra.run.smithi145.stdout:86 punch from 0xe16b000 to 0xe171200, (0x6200 bytes) 2024-09-14T07:07:52.400 INFO:teuthology.orchestra.run.smithi145.stdout:87 write 0xe73f600 thru 0xe748a4a (0x944b bytes) 2024-09-14T07:07:52.400 INFO:teuthology.orchestra.run.smithi145.stdout:88 write 0x9793600 thru 0x9797036 (0x3a37 bytes) 2024-09-14T07:07:52.400 INFO:teuthology.orchestra.run.smithi145.stdout:91 write 0x3395000 thru 0x339dc95 (0x8c96 bytes) 2024-09-14T07:07:52.402 INFO:teuthology.orchestra.run.smithi145.stdout:93 trunc from 0xed29800 to 0xdd61a00 2024-09-14T07:07:52.429 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:52.429 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:52.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:07:52.456 INFO:teuthology.orchestra.run.smithi145.stdout:96 read 0x4f35000 thru 0x4f40fcb (0xbfcc bytes) 2024-09-14T07:07:52.458 INFO:teuthology.orchestra.run.smithi145.stdout:97 punch from 0xdaddc00 to 0xdae8800, (0xac00 bytes) 2024-09-14T07:07:52.471 INFO:teuthology.orchestra.run.smithi145.stdout:98 trunc from 0xdd61a00 to 0x82c4600 2024-09-14T07:07:52.491 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:52.490+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:52.508 INFO:teuthology.orchestra.run.smithi145.stdout:99 punch from 0x42ace00 to 0x42b4a00, (0x7c00 bytes) 2024-09-14T07:07:52.510 INFO:teuthology.orchestra.run.smithi145.stdout:101 punch from 0x2aca800 to 0x2ad1000, (0x6800 bytes) 2024-09-14T07:07:52.510 INFO:teuthology.orchestra.run.smithi145.stdout:104 punch from 0x6ab3600 to 0x6abd200, (0x9c00 bytes) 2024-09-14T07:07:52.510 INFO:teuthology.orchestra.run.smithi145.stdout:105 punch from 0x3181c00 to 0x318c600, (0xaa00 bytes) 2024-09-14T07:07:52.510 INFO:teuthology.orchestra.run.smithi145.stdout:107 read 0x217b000 thru 0x2186bf7 (0xbbf8 bytes) 2024-09-14T07:07:52.522 INFO:teuthology.orchestra.run.smithi145.stdout:109 trunc from 0x82c4600 to 0x1095a00 2024-09-14T07:07:52.559 INFO:teuthology.orchestra.run.smithi145.stdout:110 read 0x6fb000 thru 0x7034ef (0x84f0 bytes) 2024-09-14T07:07:52.562 INFO:teuthology.orchestra.run.smithi145.stdout:112 read 0x4c4e00 thru 0x4c5c27 (0xe28 bytes) 2024-09-14T07:07:52.563 INFO:teuthology.orchestra.run.smithi145.stdout:113 read 0xd7000 thru 0xe07b2 (0x97b3 bytes) 2024-09-14T07:07:52.563 INFO:teuthology.orchestra.run.smithi145.stdout:115 punch from 0xfabc00 to 0xfb4a00, (0x8e00 bytes) 2024-09-14T07:07:52.564 INFO:teuthology.orchestra.run.smithi145.stdout:116 punch from 0xa65e00 to 0xa72c00, (0xce00 bytes) 2024-09-14T07:07:52.564 INFO:teuthology.orchestra.run.smithi145.stdout:117 punch from 0xf5c00 to 0x102000, (0xc400 bytes) 2024-09-14T07:07:52.564 INFO:teuthology.orchestra.run.smithi145.stdout:120 punch from 0x1eea00 to 0x1f7800, (0x8e00 bytes) 2024-09-14T07:07:52.568 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:52.568 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:52.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:07:52.574 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:52.574+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:52.582 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:07:52.582 INFO:teuthology.orchestra.run.smithi145.stdout:122 trunc from 0x1095a00 to 0xb9e4200 2024-09-14T07:07:52.609 INFO:teuthology.orchestra.run.smithi145.stdout:123 trunc from 0xb9e4200 to 0xb0cd000 2024-09-14T07:07:52.635 INFO:teuthology.orchestra.run.smithi145.stdout:126 write 0x4409400 thru 0x440dfe2 (0x4be3 bytes) 2024-09-14T07:07:52.638 INFO:teuthology.orchestra.run.smithi145.stdout:127 write 0xad0f000 thru 0xad123c6 (0x33c7 bytes) 2024-09-14T07:07:52.638 INFO:teuthology.orchestra.run.smithi145.stdout:130 punch from 0x72f0e00 to 0x72f3200, (0x2400 bytes) 2024-09-14T07:07:52.673 INFO:teuthology.orchestra.run.smithi145.stdout:132 write 0xe8df000 thru 0xe8e64f1 (0x74f2 bytes) 2024-09-14T07:07:52.675 INFO:teuthology.orchestra.run.smithi145.stdout:134 read 0x7500e00 thru 0x7501882 (0xa83 bytes) 2024-09-14T07:07:52.675 INFO:teuthology.orchestra.run.smithi145.stdout:137 read 0x8726800 thru 0x8731a75 (0xb276 bytes) 2024-09-14T07:07:52.675 INFO:teuthology.orchestra.run.smithi145.stdout:139 punch from 0xe36ee00 to 0xe372200, (0x3400 bytes) 2024-09-14T07:07:52.676 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:52.674+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:52.697 INFO:teuthology.orchestra.run.smithi145.stdout:142 trunc from 0xe8e6600 to 0x12fac00 2024-09-14T07:07:52.714 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:07:52.714 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:07:52.731 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:07:52.731 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:07:52.745 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:52.745 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:52.745 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:07:52.745 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:07:52.745 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:07:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:07:52.748 INFO:teuthology.orchestra.run.smithi145.stdout:149 trunc from 0x12fac00 to 0x6ebee00 2024-09-14T07:07:52.759 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:07:52.768 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:07:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:07:52.771 INFO:teuthology.orchestra.run.smithi145.stdout:152 read 0x993a00 thru 0x99fca0 (0xc2a1 bytes) 2024-09-14T07:07:52.776 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:52.774+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:52.790 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:07:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:07:52.810 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:07:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:07:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.2. 2024-09-14T07:07:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:07:52.828 INFO:teuthology.orchestra.run.smithi145.stdout:153 write 0xcfd3a00 thru 0xcfdd14a (0x974b bytes) 2024-09-14T07:07:52.830 INFO:teuthology.orchestra.run.smithi145.stdout:155 write 0x8e4000 thru 0x8efd10 (0xbd11 bytes) 2024-09-14T07:07:52.841 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-14T07:07:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:07:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:07:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:07:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4199.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:37.029151+0000", 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.851152951, 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015716370000000001, 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029151+0000", 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029151+0000", 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029152+0000", 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029162+0000", 2024-09-14T07:07:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3000000000000007e-06 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029189+0000", 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7653000000000001e-05 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029215+0000", 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6177e-05 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029328+0000", 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112515 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.029359+0000", 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1103999999999999e-05 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.030621+0000", 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001262186 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:37.030723+0000", 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101809 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4199.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.842382710000001, 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017608540000000001, 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037922+0000", 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-07 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037940+0000", 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8009999999999999e-05 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.037993+0000", 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2945999999999998e-05 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.038014+0000", 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0509999999999998e-05 2024-09-14T07:07:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.038166+0000", 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015273899999999999 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.038234+0000", 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7303000000000003e-05 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.039599+0000", 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013654590000000001 2024-09-14T07:07:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:38.039682+0000", 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3250000000000004e-05 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:7 3.1a e23/21 3:5cea7035:::rbd_info:head v 23'1, mlcod=0'0)", 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0815003729999999, 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021634900000000001, 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798805+0000", 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1999999999999999e-07 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798819+0000", 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798879+0000", 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9373000000000002e-05 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798902+0000", 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2836e-05 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.799063+0000", 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016089899999999999 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.799100+0000", 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7740000000000001e-05 2024-09-14T07:07:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.800872+0000", 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017718390000000001 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.800967+0000", 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5292999999999997e-05 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:49.809641+0000", 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.0706629510000001, 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001028235, 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809641+0000", 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809641+0000", 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809642+0000", 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809648+0000", 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2179999999999996e-06 2024-09-14T07:07:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809667+0000", 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8751999999999999e-05 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809679+0000", 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2510000000000001e-05 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809772+0000", 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2621000000000006e-05 2024-09-14T07:07:52.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.809803+0000", 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1409999999999999e-05 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.810622+0000", 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000818555 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:49.810670+0000", 2024-09-14T07:07:52.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7744000000000002e-05 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:876 3.19 3:9fd461fc:::rbd_data.11611b5ca054.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:52.762051+0000", 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.118253501, 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00037842099999999999, 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:07:52.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 876 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762051+0000", 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762051+0000", 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762053+0000", 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.249e-06 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762062+0000", 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4320000000000005e-06 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762097+0000", 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4938999999999999e-05 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762107+0000", 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7059999999999999e-06 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762191+0000", 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3996e-05 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762288+0000", 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7231000000000005e-05 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.762429+0000", 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014086799999999999 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:904 3.19 3:9fd461fc:::rbd_data.11611b5ca054.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:52.787277+0000", 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.093027126000000002, 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030495700000000001, 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:52.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 904 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787277+0000", 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787277+0000", 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787280+0000", 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787290+0000", 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0171e-05 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787316+0000", 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6208999999999999e-05 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787326+0000", 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9680000000000001e-06 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787401+0000", 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4963999999999996e-05 2024-09-14T07:07:52.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787473+0000", 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1827000000000002e-05 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:52.787582+0000", 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109418 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:52.905 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:52.906 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:52.878+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:52.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:07:52.994 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:52.994+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:53.074 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:53.075 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:53.075 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:07:53.075 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:53.079 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:53.078+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:53.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:07:53.180 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:53.178+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:53.256 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:07:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:07:53.257 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:53.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:07:53.315 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:53.314+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:53.382 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:53.378+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4199.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.414803771999999, 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00086957099999999997, 2024-09-14T07:07:53.437 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023565+0000", 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6800000000000001e-07 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023573+0000", 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2889999999999998e-06 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023589+0000", 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5548000000000001e-05 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023600+0000", 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0548e-05 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023677+0000", 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7967999999999996e-05 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.023707+0000", 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9475000000000002e-05 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.024381+0000", 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00067376100000000004 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:36.024434+0000", 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3613999999999999e-05 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:53.441 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:53.450 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:07:53.495 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:53.494+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:53.581 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:53.582 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:53.582 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:07:53.582 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:53.583 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:53.582+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:53.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:07:53.684 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:53.682+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:53.757 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:53.757 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:53.757 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:07:53.757 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:07:53.757 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:53.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:07:53.786 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:53.782+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:53.836 INFO:teuthology.orchestra.run.smithi145.stdout:157 write 0x98ffe00 thru 0x9902465 (0x2666 bytes) 2024-09-14T07:07:53.837 INFO:teuthology.orchestra.run.smithi145.stdout:160 write 0xbb50800 thru 0xbb5d285 (0xca86 bytes) 2024-09-14T07:07:53.837 INFO:teuthology.orchestra.run.smithi145.stdout:164 read 0x341d600 thru 0x342a0f1 (0xcaf2 bytes) 2024-09-14T07:07:53.837 INFO:teuthology.orchestra.run.smithi145.stdout:165 write 0xaa27600 thru 0xaa31db4 (0xa7b5 bytes) 2024-09-14T07:07:53.838 INFO:teuthology.orchestra.run.smithi145.stdout:166 punch from 0x6153000 to 0x6162800, (0xf800 bytes) 2024-09-14T07:07:53.839 INFO:teuthology.orchestra.run.smithi145.stdout:171 read 0x30cde00 thru 0x30db4df (0xd6e0 bytes) 2024-09-14T07:07:53.839 INFO:teuthology.orchestra.run.smithi145.stdout:172 read 0x13cc00 thru 0x148c19 (0xc01a bytes) 2024-09-14T07:07:53.840 INFO:teuthology.orchestra.run.smithi145.stdout:173 punch from 0x1c03600 to 0x1c09800, (0x6200 bytes) 2024-09-14T07:07:53.841 INFO:teuthology.orchestra.run.smithi145.stdout:174 punch from 0x846c600 to 0x8478a00, (0xc400 bytes) 2024-09-14T07:07:53.842 INFO:teuthology.orchestra.run.smithi145.stdout:175 trunc from 0xcfdd200 to 0xd8ac400 2024-09-14T07:07:53.884 INFO:teuthology.orchestra.run.smithi145.stdout:177 write 0x4ef1400 thru 0x4f0011a (0xed1b bytes) 2024-09-14T07:07:53.886 INFO:teuthology.orchestra.run.smithi145.stdout:178 read 0xa65e00 thru 0xa75cda (0xfedb bytes) 2024-09-14T07:07:53.887 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:53.882+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:53.908 INFO:teuthology.orchestra.run.smithi145.stdout:179 trunc from 0xd8ac400 to 0x6438600 2024-09-14T07:07:53.913 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:07:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:07:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:07:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.611193248999999, 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:07:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:07:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:07:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.643503+0000", 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.268194105999999, 2024-09-14T07:07:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012546646, 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 20 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643503+0000", 2024-09-14T07:07:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643503+0000", 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643506+0000", 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.627e-06 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643680+0000", 2024-09-14T07:07:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017461500000000001 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643712+0000", 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2172999999999998e-05 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643730+0000", 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7110000000000001e-05 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643880+0000", 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000150929 2024-09-14T07:07:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644102+0000", 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022136399999999999 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644478+0000", 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037612700000000003 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651054+0000", 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065762340000000002 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.655704+0000", 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046501169999999996 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.655897+0000", 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019290799999999999 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656050+0000", 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000152442 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.643770+0000", 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.267926998, 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012495286, 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643770+0000", 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643770+0000", 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643771+0000", 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3170000000000001e-06 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643927+0000", 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015602500000000001 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643944+0000", 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6745999999999999e-05 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643955+0000", 2024-09-14T07:07:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0377000000000001e-05 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.646417+0000", 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024623679999999999 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.646653+0000", 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00023595199999999999 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.647026+0000", 2024-09-14T07:07:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037282999999999999 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.655664+0000", 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0086378099999999992 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656110+0000", 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044634600000000003 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656190+0000", 2024-09-14T07:07:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9789999999999993e-05 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656265+0000", 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5724999999999997e-05 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.643981+0000", 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.267716022999998, 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012545588999999999, 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643981+0000", 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643981+0000", 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643982+0000", 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1030000000000001e-06 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644058+0000", 2024-09-14T07:07:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5346000000000007e-05 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644071+0000", 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3436e-05 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644081+0000", 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5670000000000001e-06 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.648660+0000", 2024-09-14T07:07:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045795419999999998 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.648948+0000", 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000288351 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.649331+0000", 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038227399999999999 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656299+0000", 2024-09-14T07:07:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0069682520000000003 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656391+0000", 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2120000000000006e-05 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656469+0000", 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8026000000000001e-05 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656527+0000", 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7571999999999997e-05 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.644098+0000", 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.267599020999999, 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012692985, 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 23 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644098+0000", 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644098+0000", 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644099+0000", 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6700000000000002e-07 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644240+0000", 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000140572 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644253+0000", 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3793e-05 2024-09-14T07:07:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644265+0000", 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1754e-05 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651272+0000", 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070070089999999998 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651460+0000", 2024-09-14T07:07:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000188106 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651673+0000", 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000212258 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656639+0000", 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0049660110000000002 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656669+0000", 2024-09-14T07:07:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0766000000000003e-05 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656736+0000", 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6509999999999998e-05 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656791+0000", 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5238999999999997e-05 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.644284+0000", 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.267413605000002, 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013588008, 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 24 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644284+0000", 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644284+0000", 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644285+0000", 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1230000000000001e-06 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644428+0000", 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014294500000000001 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644441+0000", 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3893000000000001e-05 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644451+0000", 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5040000000000008e-06 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.653354+0000", 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0089025779999999995 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.653581+0000", 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022751499999999999 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.653970+0000", 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038934000000000001 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656824+0000", 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028534979999999999 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657727+0000", 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00090278999999999999 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657808+0000", 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1719999999999997e-05 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657872+0000", 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3102000000000007e-05 2024-09-14T07:07:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.644470+0000", 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.267227003999999, 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014639968, 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 25 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644470+0000", 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644470+0000", 2024-09-14T07:07:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644471+0000", 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5999999999999991e-07 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644478+0000", 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6329999999999999e-06 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644488+0000", 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0349999999999999e-05 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644496+0000", 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6000000000000001e-06 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656922+0000", 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012426368 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657054+0000", 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013235499999999999 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657266+0000", 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021203300000000001 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.658289+0000", 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010222720000000001 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.658983+0000", 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00069388100000000003 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.659057+0000", 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4553000000000003e-05 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.659110+0000", 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2963000000000001e-05 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.754783+0000", 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156913944999999, 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011459977999999999, 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 43 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754783+0000", 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754783+0000", 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754784+0000", 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0899999999999999e-06 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754791+0000", 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:07:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754803+0000", 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.241e-05 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754815+0000", 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1357e-05 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.761307+0000", 2024-09-14T07:07:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064925 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.761418+0000", 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110299 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.761589+0000", 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000171025 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.765799+0000", 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042101129999999997 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.765976+0000", 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017721999999999999 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766124+0000", 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000148138 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766243+0000", 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119076 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.754969+0000", 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156728358999999, 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011519387000000001, 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 44 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754969+0000", 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754969+0000", 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754970+0000", 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755047+0000", 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7245999999999999e-05 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755062+0000", 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4929000000000001e-05 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755075+0000", 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.325e-05 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.762648+0000", 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0075726860000000003 2024-09-14T07:07:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.762760+0000", 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112248 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.762938+0000", 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017746400000000001 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.765932+0000", 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029937050000000002 2024-09-14T07:07:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766266+0000", 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033474999999999998 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766390+0000", 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012400000000000001 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766488+0000", 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7829000000000001e-05 2024-09-14T07:07:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755165+0000", 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156531768000001, 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012304374999999999, 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 45 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755165+0000", 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755165+0000", 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755167+0000", 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755176+0000", 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0170000000000002e-06 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755191+0000", 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4932999999999999e-05 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755200+0000", 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3700000000000001e-06 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.764320+0000", 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0091199490000000005 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.764550+0000", 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022994199999999999 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.764941+0000", 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00039086399999999999 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767179+0000", 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022382320000000002 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767216+0000", 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166000000000003e-05 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767345+0000", 2024-09-14T07:07:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000129176 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767470+0000", 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124509 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755218+0000", 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156479128000001, 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016364961000000001, 2024-09-14T07:07:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 46 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755218+0000", 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755218+0000", 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755219+0000", 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1999999999999998e-07 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755249+0000", 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9943000000000001e-05 2024-09-14T07:07:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755261+0000", 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1987000000000001e-05 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755269+0000", 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.123e-06 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766556+0000", 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011286608 2024-09-14T07:07:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766622+0000", 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6699000000000006e-05 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766729+0000", 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010721999999999999 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771118+0000", 2024-09-14T07:07:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043881969999999999 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771333+0000", 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000215672 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771468+0000", 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013462599999999999 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771583+0000", 2024-09-14T07:07:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114966 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755285+0000", 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156412247999999, 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016543782, 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 47 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755285+0000", 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755285+0000", 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755286+0000", 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0359999999999999e-06 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755389+0000", 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103286 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755415+0000", 2024-09-14T07:07:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6282999999999999e-05 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755424+0000", 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0340000000000002e-06 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767674+0000", 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012249711 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767816+0000", 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014178899999999999 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.768030+0000", 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021378200000000001 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771292+0000", 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032618479999999999 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771608+0000", 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031678799999999998 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771735+0000", 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012682899999999999 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771829+0000", 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3395999999999998e-05 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755459+0000", 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156237767, 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017949085, 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 48 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755459+0000", 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755459+0000", 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755461+0000", 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.184e-06 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755469+0000", 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9460000000000004e-06 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755483+0000", 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3213999999999999e-05 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755491+0000", 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0099999999999995e-06 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.769464+0000", 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013973785000000001 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.769707+0000", 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024299799999999999 2024-09-14T07:07:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.770118+0000", 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041058700000000002 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773131+0000", 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030129940000000002 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773167+0000", 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6288999999999998e-05 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773300+0000", 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013227599999999999 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773408+0000", 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010880200000000001 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156189536999999, 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022538355999999999, 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 49 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:07:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6400000000000001e-07 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755514+0000", 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2800000000000003e-06 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755527+0000", 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.341e-05 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755536+0000", 2024-09-14T07:07:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771932+0000", 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016396852999999999 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.772068+0000", 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013607300000000001 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.772298+0000", 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000229528 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.776449+0000", 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041507469999999998 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.777782+0000", 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013336839999999999 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.777921+0000", 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013824900000000001 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778046+0000", 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000125282 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755548+0000", 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.156148953999999, 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022670177, 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755548+0000", 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755548+0000", 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755549+0000", 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1699999999999997e-07 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755644+0000", 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4913000000000004e-05 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755664+0000", 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0259999999999999e-05 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755678+0000", 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3949e-05 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773598+0000", 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017920373999999999 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773773+0000", 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017479600000000001 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773990+0000", 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021722700000000001 2024-09-14T07:07:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778071+0000", 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0040802160000000002 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778116+0000", 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5216999999999998e-05 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778162+0000", 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5775999999999997e-05 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778218+0000", 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6632000000000003e-05 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755705+0000", 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.155992321999999, 2024-09-14T07:07:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022896933000000001, 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 51 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755705+0000", 2024-09-14T07:07:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755705+0000", 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755706+0000", 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5239999999999999e-06 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755718+0000", 2024-09-14T07:07:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1513e-05 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755736+0000", 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8199999999999999e-05 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755750+0000", 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3986e-05 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.774878+0000", 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.019128432000000001 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.775122+0000", 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024310799999999999 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.775531+0000", 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00040984099999999998 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778327+0000", 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027959479999999999 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778363+0000", 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5790000000000001e-05 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778495+0000", 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131409 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778602+0000", 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107182 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755801+0000", 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.155896039000002, 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.023029786, 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 52 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755801+0000", 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755801+0000", 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755802+0000", 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.367e-06 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755813+0000", 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.059e-05 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755841+0000", 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7662e-05 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755858+0000", 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7640000000000001e-05 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.776554+0000", 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.020695344000000001 2024-09-14T07:07:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.776805+0000", 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025167100000000002 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.777185+0000", 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037977699999999998 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778627+0000", 2024-09-14T07:07:54.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001441559 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778736+0000", 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108942 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778779+0000", 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2926999999999997e-05 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778831+0000", 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2306999999999999e-05 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.958068+0000", 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 20.953628634000001, 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010754107000000001, 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 74 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958068+0000", 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958068+0000", 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958069+0000", 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8000000000000005e-07 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958074+0000", 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0810000000000001e-06 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958083+0000", 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9269999999999994e-06 2024-09-14T07:07:54.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958090+0000", 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6470000000000001e-06 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.965815+0000", 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077252459999999998 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.965871+0000", 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5779000000000002e-05 2024-09-14T07:07:54.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.965963+0000", 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2251999999999997e-05 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967977+0000", 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020139450000000001 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.968755+0000", 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00077767299999999999 2024-09-14T07:07:54.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.968794+0000", 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9168000000000002e-05 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.968823+0000", 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8609e-05 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.958152+0000", 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 20.953544889, 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011192554, 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 75 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958152+0000", 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958152+0000", 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958153+0000", 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.046e-06 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958261+0000", 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958276+0000", 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4656e-05 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958286+0000", 2024-09-14T07:07:54.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4809999999999995e-06 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.966282+0000", 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0079967140000000003 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.966345+0000", 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3138000000000004e-05 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.966442+0000", 2024-09-14T07:07:54.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6281999999999997e-05 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967983+0000", 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015415240000000001 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969293+0000", 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00130934 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969323+0000", 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0306000000000001e-05 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969345+0000", 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.192e-05 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.958307+0000", 2024-09-14T07:07:54.025 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 20.953390220999999, 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011418697, 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 76 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958307+0000", 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958307+0000", 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958308+0000", 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6600000000000005e-07 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958313+0000", 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5419999999999996e-06 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958322+0000", 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958354+0000", 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2017999999999997e-05 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967384+0000", 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.009029825 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967440+0000", 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6264999999999997e-05 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967541+0000", 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100713 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.968204+0000", 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066269600000000005 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969666+0000", 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001462472 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969696+0000", 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9703999999999999e-05 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969726+0000", 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9759000000000001e-05 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.030 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.031 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.031 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:54.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:07:54.033 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:54.030+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:180 read 0x2560c00 thru 0x2561fa9 (0x13aa bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:181 read 0xa19000 thru 0xa28e43 (0xfe44 bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:184 write 0xce6d400 thru 0xce702c5 (0x2ec6 bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:187 write 0xce8e200 thru 0xce8e480 (0x281 bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:188 write 0x88d9c00 thru 0x88e475b (0xab5c bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:189 write 0x70c4c00 thru 0x70d3a30 (0xee31 bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:193 punch from 0x6c8a000 to 0x6c95000, (0xb000 bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:194 write 0x50b6a00 thru 0x50bb626 (0x4c27 bytes) 2024-09-14T07:07:54.034 INFO:teuthology.orchestra.run.smithi145.stdout:197 write 0xbfd2600 thru 0xbfe0eff (0xe900 bytes) 2024-09-14T07:07:54.035 INFO:teuthology.orchestra.run.smithi145.stdout:198 read 0x891ba00 thru 0x892b43f (0xfa40 bytes) 2024-09-14T07:07:54.035 INFO:teuthology.orchestra.run.smithi145.stdout:199 write 0xa925a00 thru 0xa92d0c5 (0x76c6 bytes) 2024-09-14T07:07:54.035 INFO:teuthology.orchestra.run.smithi145.stdout:200 write 0xb75d400 thru 0xb7619de (0x45df bytes) 2024-09-14T07:07:54.035 INFO:teuthology.orchestra.run.smithi145.stdout:204 trunc from 0xce8e600 to 0xa14b200 2024-09-14T07:07:54.103 INFO:teuthology.orchestra.run.smithi145.stdout:205 write 0xd3e2e00 thru 0xd3ef22f (0xc430 bytes) 2024-09-14T07:07:54.104 INFO:teuthology.orchestra.run.smithi145.stdout:206 read 0x67f5200 thru 0x6804e3e (0xfc3f bytes) 2024-09-14T07:07:54.110 INFO:teuthology.orchestra.run.smithi145.stdout:207 trunc from 0xd3ef400 to 0xb4aec00 2024-09-14T07:07:54.133 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:54.130+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:54.148 INFO:teuthology.orchestra.run.smithi145.stdout:209 trunc from 0xb4aec00 to 0xbc42a00 2024-09-14T07:07:54.165 INFO:teuthology.orchestra.run.smithi145.stdout:213 write 0x2770800 thru 0x27754b5 (0x4cb6 bytes) 2024-09-14T07:07:54.166 INFO:teuthology.orchestra.run.smithi145.stdout:214 write 0x886a600 thru 0x88737b3 (0x91b4 bytes) 2024-09-14T07:07:54.178 INFO:teuthology.orchestra.run.smithi145.stdout:215 trunc from 0xbc42a00 to 0x7d57000 2024-09-14T07:07:54.217 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:54.217 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:54.217 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:07:54.217 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:54.219 INFO:teuthology.orchestra.run.smithi145.stdout:219 trunc from 0x7d57000 to 0x6fb7a00 2024-09-14T07:07:54.233 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:07:54.233 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:54.230+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:54.249 INFO:teuthology.orchestra.run.smithi145.stdout:220 punch from 0x1fde000 to 0x1feaa00, (0xca00 bytes) 2024-09-14T07:07:54.251 INFO:teuthology.orchestra.run.smithi145.stdout:222 punch from 0x25f3e00 to 0x25fc800, (0x8a00 bytes) 2024-09-14T07:07:54.251 INFO:teuthology.orchestra.run.smithi145.stdout:227 read 0x61e1800 thru 0x61e5721 (0x3f22 bytes) 2024-09-14T07:07:54.253 INFO:teuthology.orchestra.run.smithi145.stdout:231 trunc from 0x6fb7a00 to 0x640a000 2024-09-14T07:07:54.294 INFO:teuthology.orchestra.run.smithi145.stdout:233 trunc from 0x640a000 to 0xa5fc200 2024-09-14T07:07:54.313 INFO:teuthology.orchestra.run.smithi145.stdout:234 write 0x4f0d800 thru 0x4f12b3e (0x533f bytes) 2024-09-14T07:07:54.318 INFO:teuthology.orchestra.run.smithi145.stdout:235 punch from 0x5370000 to 0x537e800, (0xe800 bytes) 2024-09-14T07:07:54.318 INFO:teuthology.orchestra.run.smithi145.stdout:236 read 0xbcd800 thru 0xbd7b3b (0xa33c bytes) 2024-09-14T07:07:54.319 INFO:teuthology.orchestra.run.smithi145.stdout:240 read 0x2cc0000 thru 0x2cca50b (0xa50c bytes) 2024-09-14T07:07:54.320 INFO:teuthology.orchestra.run.smithi145.stdout:242 read 0x1aa8a00 thru 0x1ab75ab (0xebac bytes) 2024-09-14T07:07:54.334 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:54.330+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:54.343 INFO:teuthology.orchestra.run.smithi145.stdout:244 write 0xd783000 thru 0xd78ce11 (0x9e12 bytes) 2024-09-14T07:07:54.345 INFO:teuthology.orchestra.run.smithi145.stdout:245 write 0x6ff4200 thru 0x70017a0 (0xd5a1 bytes) 2024-09-14T07:07:54.345 INFO:teuthology.orchestra.run.smithi145.stdout:247 read 0x1064800 thru 0x1065827 (0x1028 bytes) 2024-09-14T07:07:54.346 INFO:teuthology.orchestra.run.smithi145.stdout:251 read 0x1681000 thru 0x168eb77 (0xdb78 bytes) 2024-09-14T07:07:54.347 INFO:teuthology.orchestra.run.smithi145.stdout:252 punch from 0x6c92c00 to 0x6ca1400, (0xe800 bytes) 2024-09-14T07:07:54.349 INFO:teuthology.orchestra.run.smithi145.stdout:253 trunc from 0xd78d000 to 0xe322200 2024-09-14T07:07:54.379 INFO:teuthology.orchestra.run.smithi145.stdout:254 read 0x71bea00 thru 0x71cae12 (0xc413 bytes) 2024-09-14T07:07:54.390 INFO:teuthology.orchestra.run.smithi145.stdout:255 trunc from 0xe322200 to 0x9f47600 2024-09-14T07:07:54.413 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:54.413 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:54.413 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:07:54.413 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:07:54.414 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:54.420 INFO:teuthology.orchestra.run.smithi145.stdout:256 read 0x2d05600 thru 0x2d1164b (0xc04c bytes) 2024-09-14T07:07:54.428 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:07:54.455 INFO:teuthology.orchestra.run.smithi145.stdout:257 write 0xe475600 thru 0xe47aee2 (0x58e3 bytes) 2024-09-14T07:07:54.456 INFO:teuthology.orchestra.run.smithi145.stdout:258 read 0x78ed400 thru 0x78fa1f4 (0xcdf5 bytes) 2024-09-14T07:07:54.456 INFO:teuthology.orchestra.run.smithi145.stdout:259 read 0x1424a00 thru 0x1424e32 (0x433 bytes) 2024-09-14T07:07:54.457 INFO:teuthology.orchestra.run.smithi145.stdout:261 clone 1 order 25 su 524288 sc 15 2024-09-14T07:07:54.471 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:54.470+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:54.536 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:54.534+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:54.552 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:54.552 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:07:54.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:07:54.552 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.7555325770000003, 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070947240000000002, 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 7 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.553 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793553+0000", 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3730000000000001e-06 2024-09-14T07:07:54.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793583+0000", 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9675999999999998e-05 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793621+0000", 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7370000000000003e-05 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793647+0000", 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6276999999999999e-05 2024-09-14T07:07:54.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793835+0000", 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018812500000000001 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793852+0000", 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.734e-05 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797434+0000", 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035819189999999998 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797559+0000", 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012508500000000001 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797726+0000", 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000166797 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.798685+0000", 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00095928199999999999 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800546+0000", 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018609189999999999 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800602+0000", 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5365999999999998e-05 2024-09-14T07:07:54.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800645+0000", 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3195000000000002e-05 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.748099066, 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0097102179999999996, 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.559 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800987+0000", 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801000+0000", 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2897e-05 2024-09-14T07:07:54.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801028+0000", 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8575999999999999e-05 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801054+0000", 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6367e-05 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801282+0000", 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022778100000000001 2024-09-14T07:07:54.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801300+0000", 2024-09-14T07:07:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7237000000000001e-05 2024-09-14T07:07:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808053+0000", 2024-09-14T07:07:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0067535249999999998 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808316+0000", 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000263224 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808688+0000", 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037128699999999998 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810367+0000", 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001679151 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810427+0000", 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0338999999999999e-05 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810572+0000", 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014462200000000001 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810694+0000", 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000122192 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:929 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.70951231400000003, 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013302158999999999, 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.567 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 929 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839572+0000", 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6780000000000001e-06 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839580+0000", 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0989999999999999e-06 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839590+0000", 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839598+0000", 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3699999999999997e-06 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840130+0000", 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00053217 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840818+0000", 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00068814799999999997 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845641+0000", 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048226160000000001 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845730+0000", 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9687999999999999e-05 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846136+0000", 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000405583 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849695+0000", 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00355887 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852725+0000", 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030301009999999999 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852786+0000", 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1081999999999996e-05 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852873+0000", 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6608999999999997e-05 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:930 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.70947273200000005, 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013501599, 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 930 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.574 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:54.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839613+0000", 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4169999999999999e-06 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839618+0000", 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1499999999999998e-06 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839622+0000", 2024-09-14T07:07:54.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.631e-06 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841189+0000", 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001567768 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841262+0000", 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2105000000000002e-05 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846495+0000", 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052335999999999997 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846559+0000", 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3609999999999996e-05 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846782+0000", 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022373299999999999 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849705+0000", 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002922998 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852940+0000", 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032343070000000001 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853014+0000", 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3967999999999994e-05 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853112+0000", 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7806000000000005e-05 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:931 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.70940442699999995, 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.01360341, 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.580 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 931 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839679+0000", 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7299999999999996e-07 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839682+0000", 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839688+0000", 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9569999999999999e-06 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839693+0000", 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6289999999999998e-06 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841324+0000", 2024-09-14T07:07:54.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016306090000000001 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841425+0000", 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010129299999999999 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847149+0000", 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057235139999999999 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847194+0000", 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5117000000000002e-05 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847441+0000", 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000247428 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852669+0000", 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052276029999999999 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853162+0000", 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049292300000000004 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853202+0000", 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0525999999999998e-05 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853282+0000", 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9513999999999998e-05 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:932 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.70929760500000005, 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013718054, 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 932 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.587 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839786+0000", 2024-09-14T07:07:54.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4499999999999996e-07 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839794+0000", 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6450000000000005e-06 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839805+0000", 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1489e-05 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839814+0000", 2024-09-14T07:07:54.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6540000000000005e-06 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841579+0000", 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017656480000000001 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841669+0000", 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9806e-05 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847963+0000", 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0062937080000000003 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848135+0000", 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017191299999999999 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848791+0000", 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00065584300000000003 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853128+0000", 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043374149999999998 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853351+0000", 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000223408 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853408+0000", 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6922000000000002e-05 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853503+0000", 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4758000000000003e-05 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:933 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.70886665100000001, 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015741667000000001, 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 933 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840217+0000", 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1909999999999999e-06 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840230+0000", 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3226e-05 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840244+0000", 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3108999999999999e-05 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840252+0000", 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2910000000000007e-06 2024-09-14T07:07:54.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841706+0000", 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001454056 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841766+0000", 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0494000000000001e-05 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849824+0000", 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080573079999999991 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850028+0000", 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020432999999999999 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850748+0000", 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072041399999999995 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853135+0000", 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002386885 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855783+0000", 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026479350000000001 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855851+0000", 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7534999999999996e-05 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855958+0000", 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010689299999999999 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:935 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.70715751299999996, 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016747201, 2024-09-14T07:07:54.600 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 935 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:07:54.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841926+0000", 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841932+0000", 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2099999999999998e-06 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841943+0000", 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0981e-05 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841950+0000", 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842004+0000", 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4422000000000002e-05 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842064+0000", 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9967999999999999e-05 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851737+0000", 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0096733600000000006 2024-09-14T07:07:54.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851825+0000", 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7855999999999994e-05 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852258+0000", 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000432383 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853520+0000", 2024-09-14T07:07:54.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001262787 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858562+0000", 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050418269999999996 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858623+0000", 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0789999999999999e-05 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858672+0000", 2024-09-14T07:07:54.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9348999999999998e-05 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:936 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.70691575500000003, 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.018202472000000001, 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 936 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:07:54.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842168+0000", 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3300000000000002e-07 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842170+0000", 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6539999999999999e-06 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842178+0000", 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.413e-06 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842182+0000", 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3139999999999997e-06 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842231+0000", 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8553000000000001e-05 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842290+0000", 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9534000000000003e-05 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853603+0000", 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011312579 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853796+0000", 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299000000000001 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854842+0000", 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00104612 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858690+0000", 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038478589999999999 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860251+0000", 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015616829999999999 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860315+0000", 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3587e-05 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860369+0000", 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4552999999999998e-05 2024-09-14T07:07:54.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:937 3.11 e26/21 3:8ce9c1c4:::rbd_data.11611b5ca054.0000000000000026:head v 26'3, mlcod=24'1)", 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.853952+0000", 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.69513036299999997, 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047661400000000003, 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853952+0000", 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853952+0000", 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853954+0000", 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:07:54.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853987+0000", 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3602000000000001e-05 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854007+0000", 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9806e-05 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854021+0000", 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3757999999999999e-05 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855996+0000", 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019747599999999999 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856019+0000", 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.298e-05 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858693+0000", 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026738249999999999 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858718+0000", 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950999999999999e-05 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.618 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1141 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.016926+0000", 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.53215657599999999, 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006443926, 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1141 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016926+0000", 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016926+0000", 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016928+0000", 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7039999999999999e-06 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016937+0000", 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-06 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016961+0000", 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4241000000000001e-05 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016994+0000", 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2335999999999997e-05 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017103+0000", 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109522 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017308+0000", 2024-09-14T07:07:54.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020519900000000001 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.018237+0000", 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00092822100000000002 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023056+0000", 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048193039999999999 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023174+0000", 2024-09-14T07:07:54.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118393 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023271+0000", 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6446000000000005e-05 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023370+0000", 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9291999999999997e-05 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1142 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.53203755699999999, 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.007319173, 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1142 2024-09-14T07:07:54.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017046+0000", 2024-09-14T07:07:54.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017061+0000", 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.467e-05 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017076+0000", 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5421e-05 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017088+0000", 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1287999999999999e-05 2024-09-14T07:07:54.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019464+0000", 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023761590000000001 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019645+0000", 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000180772 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.020444+0000", 2024-09-14T07:07:54.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00079905700000000004 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023077+0000", 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002633093 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024178+0000", 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011013850000000001 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024296+0000", 2024-09-14T07:07:54.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117586 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024364+0000", 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8585999999999997e-05 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1145 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.53173923700000003, 2024-09-14T07:07:54.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077736239999999998, 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1145 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017346+0000", 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2369999999999999e-06 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017358+0000", 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2602999999999999e-05 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017390+0000", 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.188e-05 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017413+0000", 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.243e-05 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021308+0000", 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038951799999999998 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021511+0000", 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020368099999999999 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.022262+0000", 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00075023799999999997 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023822+0000", 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015606750000000001 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025018+0000", 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001195149 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8764999999999997e-05 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025117+0000", 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0785999999999999e-05 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.636 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1159 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.52411381300000004, 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0074537930000000002, 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1159 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024970+0000", 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5939999999999999e-06 2024-09-14T07:07:54.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024981+0000", 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025004+0000", 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3468999999999998e-05 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025021+0000", 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6674e-05 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025194+0000", 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173721 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025445+0000", 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025037099999999999 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.026330+0000", 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000885073 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032132+0000", 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058019819999999998 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032250+0000", 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118447 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032351+0000", 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100966 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032423+0000", 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1351000000000002e-05 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.642 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1160 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.52400787999999998, 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0076908269999999999, 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.643 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1160 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.251e-06 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025084+0000", 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5689999999999999e-06 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025102+0000", 2024-09-14T07:07:54.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8734000000000001e-05 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025116+0000", 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3465999999999999e-05 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027183+0000", 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002066877 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027355+0000", 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017235800000000001 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028074+0000", 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00071921500000000005 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032586+0000", 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045120960000000002 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032617+0000", 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0468999999999999e-05 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032697+0000", 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9939999999999997e-05 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032766+0000", 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8851999999999997e-05 2024-09-14T07:07:54.648 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1161 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.52390967499999996, 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0078300520000000005, 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1161 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025174+0000", 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4139999999999999e-06 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025184+0000", 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3489999999999998e-06 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025201+0000", 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7011e-05 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025214+0000", 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3237e-05 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028886+0000", 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036715649999999999 2024-09-14T07:07:54.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029039+0000", 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015298300000000001 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029611+0000", 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057197599999999997 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032782+0000", 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031712340000000002 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032863+0000", 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0907000000000003e-05 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032957+0000", 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4715000000000004e-05 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033003+0000", 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5661000000000002e-05 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1162 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.523842013, 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084900349999999999, 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.655 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1162 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025242+0000", 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3829999999999999e-06 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025248+0000", 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1999999999999999e-06 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025283+0000", 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4347000000000002e-05 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025295+0000", 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2268e-05 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030146+0000", 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048513200000000001 2024-09-14T07:07:54.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030309+0000", 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000163225 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.031281+0000", 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00097148600000000003 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033053+0000", 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001771941 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033582+0000", 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052929400000000003 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033653+0000", 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1285999999999995e-05 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033731+0000", 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7285000000000003e-05 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1167 3.a e26/21 3:509716c6:::rbd_data.11611b5ca054.000000000000002a:head v 26'12, mlcod=26'10)", 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.51496942300000004, 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065049039999999997, 2024-09-14T07:07:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034114+0000", 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0800000000000003e-07 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034143+0000", 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8532000000000001e-05 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034158+0000", 2024-09-14T07:07:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5525000000000001e-05 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034169+0000", 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0726e-05 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034296+0000", 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012745700000000001 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034352+0000", 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5229999999999998e-05 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040532+0000", 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0061802130000000004 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040618+0000", 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6312999999999998e-05 2024-09-14T07:07:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1586 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.348680+0000", 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.20040221499999999, 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042129139999999999, 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1586 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348680+0000", 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348680+0000", 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348681+0000", 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4700000000000001e-07 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348686+0000", 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3900000000000003e-06 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348700+0000", 2024-09-14T07:07:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4388e-05 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348707+0000", 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1679999999999998e-06 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.349668+0000", 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096081900000000002 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.349851+0000", 2024-09-14T07:07:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018299200000000001 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.350291+0000", 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044031700000000002 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.351782+0000", 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014911620000000001 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352786+0000", 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010035409999999999 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352850+0000", 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3953000000000002e-05 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352893+0000", 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3436999999999998e-05 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1587 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.348720+0000", 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.20036282699999999, 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051567840000000002, 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1587 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348720+0000", 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348720+0000", 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348721+0000", 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7299999999999995e-07 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348727+0000", 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0399999999999998e-06 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348735+0000", 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5299999999999996e-06 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348742+0000", 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6769999999999998e-06 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.350868+0000", 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021257490000000001 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.350986+0000", 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011850599999999999 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.351385+0000", 2024-09-14T07:07:54.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00039872699999999998 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352424+0000", 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010396590000000001 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.353783+0000", 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013586049999999999 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.353836+0000", 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2787999999999997e-05 2024-09-14T07:07:54.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.353877+0000", 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.083e-05 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.677 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:54.679 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:07:54.679 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:54.674+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:54.779 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:54.778+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:54.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:07:54.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1778 3.1d 3:b8c07fa4:::rbd_header.11611b5ca054:head [call rbd.parent_get in=13b out=30b,call rbd.parent_overlap_get in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.855023+0000", 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0010273470000000001, 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010930289999999999, 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:54.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1778 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855023+0000", 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855023+0000", 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855024+0000", 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5430000000000001e-06 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855030+0000", 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855044+0000", 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4061e-05 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855053+0000", 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8759999999999994e-06 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855476+0000", 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042245100000000001 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855612+0000", 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000136675 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:604 3.1d 3.25fe031d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.855877+0000", 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00017312600000000001, 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000450297, 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 604 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.860 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855877+0000", 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855877+0000", 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855878+0000", 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.387e-06 2024-09-14T07:07:54.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855885+0000", 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855900+0000", 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5665999999999999e-05 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.855910+0000", 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9399999999999997e-06 2024-09-14T07:07:54.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.863 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:54.863 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.863 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:54.863 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:07:54.863 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 2 2024-09-14T07:07:54.863 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:54.872 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:07:54.919 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:54.918+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:54.981 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:54.978+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:54.996 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:54.996 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:54.996 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:07:54.996 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:07:54.996 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:55.001 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:07:55.001 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:07:55.011 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:07:55.020 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:07:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:07:55.037 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:07:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:07:55.057 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:07:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:07:55.075 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:07:55.075 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:07:55.096 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:07:55.096 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:07:55.099 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:55.098+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:55.182 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:55.178+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:55.184 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:55.184 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:55.200 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:07:55.283 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:55.282+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:55.322 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:55.323 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:55.337 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:07:55.352 INFO:teuthology.orchestra.run.smithi145.stdout:263 trunc from 0xe47b000 to 0xc315e00 2024-09-14T07:07:55.389 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:55.382+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:55.422 INFO:teuthology.orchestra.run.smithi145.stdout:264 write 0xab75600 thru 0xab81b22 (0xc523 bytes) 2024-09-14T07:07:55.426 INFO:teuthology.orchestra.run.smithi145.stdout:265 read 0xcf4600 thru 0xcf4c15 (0x616 bytes) 2024-09-14T07:07:55.427 INFO:teuthology.orchestra.run.smithi145.stdout:268 punch from 0x1a0a200 to 0x1a12e00, (0x8c00 bytes) 2024-09-14T07:07:55.428 INFO:teuthology.orchestra.run.smithi145.stdout:269 read 0xbb92800 thru 0xbb9596a (0x316b bytes) 2024-09-14T07:07:55.429 INFO:teuthology.orchestra.run.smithi145.stdout:270 punch from 0x2829200 to 0x2832200, (0x9000 bytes) 2024-09-14T07:07:55.461 INFO:teuthology.orchestra.run.smithi145.stdout:272 write 0xccdf200 thru 0xccec243 (0xd044 bytes) 2024-09-14T07:07:55.467 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:55.467 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:55.468 INFO:teuthology.orchestra.run.smithi145.stdout:273 flatten 2024-09-14T07:07:55.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:07:55.501 INFO:teuthology.orchestra.run.smithi145.stdout:278 punch from 0x4dad800 to 0x4dbac00, (0xd400 bytes) 2024-09-14T07:07:55.504 INFO:teuthology.orchestra.run.smithi145.stdout:279 write 0xa45ae00 thru 0xa45ff22 (0x5123 bytes) 2024-09-14T07:07:55.511 INFO:teuthology.orchestra.run.smithi145.stdout:280 trunc from 0xccec400 to 0xe3eca00 2024-09-14T07:07:55.527 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:55.526+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:55.537 INFO:teuthology.orchestra.run.smithi145.stdout:282 punch from 0x3167800 to 0x3176000, (0xe800 bytes) 2024-09-14T07:07:55.541 INFO:teuthology.orchestra.run.smithi145.stdout:283 write 0x9740600 thru 0x974e714 (0xe115 bytes) 2024-09-14T07:07:55.542 INFO:teuthology.orchestra.run.smithi145.stdout:285 write 0xb33c00 thru 0xb41581 (0xd982 bytes) 2024-09-14T07:07:55.545 INFO:teuthology.orchestra.run.smithi145.stdout:286 write 0x4c02600 thru 0x4c0708c (0x4a8d bytes) 2024-09-14T07:07:55.586 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:55.586+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:55.651 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:55.651 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:55.665 INFO:teuthology.orchestra.run.smithi145.stdout:288 trunc from 0xe3eca00 to 0x7a9c00 2024-09-14T07:07:55.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:07:55.687 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:55.686+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:55.743 INFO:teuthology.orchestra.run.smithi145.stdout:289 read 0x1cd200 thru 0x1d1973 (0x4774 bytes) 2024-09-14T07:07:55.748 INFO:teuthology.orchestra.run.smithi145.stdout:290 punch from 0x1a2800 to 0x1b0800, (0xe000 bytes) 2024-09-14T07:07:55.773 INFO:teuthology.orchestra.run.smithi145.stdout:291 trunc from 0x7a9c00 to 0xcb18600 2024-09-14T07:07:55.789 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:55.786+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:55.799 INFO:teuthology.orchestra.run.smithi145.stdout:294 write 0x60b9e00 thru 0x60bab1b (0xd1c bytes) 2024-09-14T07:07:55.803 INFO:teuthology.orchestra.run.smithi145.stdout:295 punch from 0x9e4bc00 to 0x9e58200, (0xc600 bytes) 2024-09-14T07:07:55.804 INFO:teuthology.orchestra.run.smithi145.stdout:296 read 0xa408200 thru 0xa416d20 (0xeb21 bytes) 2024-09-14T07:07:55.808 INFO:teuthology.orchestra.run.smithi145.stdout:298 punch from 0xc9d0000 to 0xc9d5e00, (0x5e00 bytes) 2024-09-14T07:07:55.832 INFO:teuthology.orchestra.run.smithi145.stdout:300 trunc from 0xcb18600 to 0x2b7d800 2024-09-14T07:07:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:07:55.838 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:55.838 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:55.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:07:55.889 INFO:teuthology.orchestra.run.smithi145.stdout:302 read 0xa53800 thru 0xa548c5 (0x10c6 bytes) 2024-09-14T07:07:55.893 INFO:teuthology.orchestra.run.smithi145.stdout:305 read 0x1fc4800 thru 0x1fc4d1c (0x51d bytes) 2024-09-14T07:07:55.894 INFO:teuthology.orchestra.run.smithi145.stdout:306 read 0x2661400 thru 0x2666e51 (0x5a52 bytes) 2024-09-14T07:07:55.902 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:55.902+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:55.914 INFO:teuthology.orchestra.run.smithi145.stdout:307 trunc from 0x2b7d800 to 0xa05d000 2024-09-14T07:07:55.936 INFO:teuthology.orchestra.run.smithi145.stdout:308 read 0x95bd200 thru 0x95c1c74 (0x4a75 bytes) 2024-09-14T07:07:55.966 INFO:teuthology.orchestra.run.smithi145.stdout:310 write 0xb53f600 thru 0xb548302 (0x8d03 bytes) 2024-09-14T07:07:55.971 INFO:teuthology.orchestra.run.smithi145.stdout:313 punch from 0x7f75200 to 0x7f7a800, (0x5600 bytes) 2024-09-14T07:07:55.972 INFO:teuthology.orchestra.run.smithi145.stdout:315 punch from 0xaf42400 to 0xaf4fa00, (0xd600 bytes) 2024-09-14T07:07:55.972 INFO:teuthology.orchestra.run.smithi145.stdout:316 read 0x9cec200 thru 0x9cf68b6 (0xa6b7 bytes) 2024-09-14T07:07:55.973 INFO:teuthology.orchestra.run.smithi145.stdout:318 read 0x3786c00 thru 0x379015a (0x955b bytes) 2024-09-14T07:07:55.974 INFO:teuthology.orchestra.run.smithi145.stdout:319 write 0x9e6000 thru 0x9eaf63 (0x4f64 bytes) 2024-09-14T07:07:55.975 INFO:teuthology.orchestra.run.smithi145.stdout:320 read 0x919cc00 thru 0x91ac7d2 (0xfbd3 bytes) 2024-09-14T07:07:55.975 INFO:teuthology.orchestra.run.smithi145.stdout:321 read 0x3d0c800 thru 0x3d1b91a (0xf11b bytes) 2024-09-14T07:07:55.976 INFO:teuthology.orchestra.run.smithi145.stdout:322 read 0x6b15e00 thru 0x6b1a51d (0x471e bytes) 2024-09-14T07:07:55.977 INFO:teuthology.orchestra.run.smithi145.stdout:323 write 0x631dc00 thru 0x63244af (0x68b0 bytes) 2024-09-14T07:07:55.978 INFO:teuthology.orchestra.run.smithi145.stdout:324 punch from 0xa720800 to 0xa72b800, (0xb000 bytes) 2024-09-14T07:07:55.989 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:55.989 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:55.990 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:55.990+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:56.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:07:56.007 INFO:teuthology.orchestra.run.smithi145.stdout:326 write 0xe08d800 thru 0xe0939ed (0x61ee bytes) 2024-09-14T07:07:56.012 INFO:teuthology.orchestra.run.smithi145.stdout:327 read 0x5f24600 thru 0x5f343b0 (0xfdb1 bytes) 2024-09-14T07:07:56.012 INFO:teuthology.orchestra.run.smithi145.stdout:329 read 0xbdb6200 thru 0xbdb7c1f (0x1a20 bytes) 2024-09-14T07:07:56.013 INFO:teuthology.orchestra.run.smithi145.stdout:330 read 0x6ebae00 thru 0x6ec0fcd (0x61ce bytes) 2024-09-14T07:07:56.015 INFO:teuthology.orchestra.run.smithi145.stdout:333 read 0x8c22400 thru 0x8c27619 (0x521a bytes) 2024-09-14T07:07:56.015 INFO:teuthology.orchestra.run.smithi145.stdout:337 punch from 0x1748a00 to 0x174fe00, (0x7400 bytes) 2024-09-14T07:07:56.015 INFO:teuthology.orchestra.run.smithi145.stdout:339 write 0x5430800 thru 0x543f730 (0xef31 bytes) 2024-09-14T07:07:56.016 INFO:teuthology.orchestra.run.smithi145.stdout:340 read 0x7967400 thru 0x79702c2 (0x8ec3 bytes) 2024-09-14T07:07:56.052 INFO:teuthology.orchestra.run.smithi145.stdout:341 write 0xe961a00 thru 0xe96729e (0x589f bytes) 2024-09-14T07:07:56.073 INFO:teuthology.orchestra.run.smithi145.stdout:343 trunc from 0xe967400 to 0x9b4a400 2024-09-14T07:07:56.091 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:56.090+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:56.109 INFO:teuthology.orchestra.run.smithi145.stdout:344 read 0x988aa00 thru 0x9896ee0 (0xc4e1 bytes) 2024-09-14T07:07:56.114 INFO:teuthology.orchestra.run.smithi145.stdout:346 read 0x8a0e800 thru 0x8a16aa8 (0x82a9 bytes) 2024-09-14T07:07:56.115 INFO:teuthology.orchestra.run.smithi145.stdout:347 read 0x50a5c00 thru 0x50b2626 (0xca27 bytes) 2024-09-14T07:07:56.116 INFO:teuthology.orchestra.run.smithi145.stdout:348 punch from 0x8170800 to 0x817ba00, (0xb200 bytes) 2024-09-14T07:07:56.116 INFO:teuthology.orchestra.run.smithi145.stdout:349 write 0x128dc00 thru 0x1294313 (0x6714 bytes) 2024-09-14T07:07:56.118 INFO:teuthology.orchestra.run.smithi145.stdout:350 read 0x702e200 thru 0x703d029 (0xee2a bytes) 2024-09-14T07:07:56.119 INFO:teuthology.orchestra.run.smithi145.stdout:351 write 0x8eb4400 thru 0x8ec15a3 (0xd1a4 bytes) 2024-09-14T07:07:56.122 INFO:teuthology.orchestra.run.smithi145.stdout:352 read 0x239e000 thru 0x23aa869 (0xc86a bytes) 2024-09-14T07:07:56.163 INFO:teuthology.orchestra.run.smithi145.stdout:353 write 0xc91b200 thru 0xc927cbe (0xcabf bytes) 2024-09-14T07:07:56.168 INFO:teuthology.orchestra.run.smithi145.stdout:355 write 0xb70c800 thru 0xb71c593 (0xfd94 bytes) 2024-09-14T07:07:56.170 INFO:teuthology.orchestra.run.smithi145.stdout:358 read 0x6e4d400 thru 0x6e4e398 (0xf99 bytes) 2024-09-14T07:07:56.175 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:56.175 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:56.181 INFO:teuthology.orchestra.run.smithi145.stdout:359 trunc from 0xc927e00 to 0x66d7600 2024-09-14T07:07:56.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:07:56.192 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:56.190+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:56.232 INFO:teuthology.orchestra.run.smithi145.stdout:361 punch from 0x7a2a00 to 0x7b2000, (0xf600 bytes) 2024-09-14T07:07:56.247 INFO:teuthology.orchestra.run.smithi145.stdout:362 trunc from 0x66d7600 to 0xb3be200 2024-09-14T07:07:56.269 INFO:teuthology.orchestra.run.smithi145.stdout:363 write 0x4e56000 thru 0x4e65c8e (0xfc8f bytes) 2024-09-14T07:07:56.293 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:56.290+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:56.311 INFO:teuthology.orchestra.run.smithi145.stdout:366 write 0xc549000 thru 0xc5530cc (0xa0cd bytes) 2024-09-14T07:07:56.324 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:56.324 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:56.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:07:56.345 INFO:teuthology.orchestra.run.smithi145.stdout:368 trunc from 0xc553200 to 0x1487000 2024-09-14T07:07:56.394 INFO:teuthology.orchestra.run.smithi145.stdout:373 trunc from 0x1487000 to 0x19a1600 2024-09-14T07:07:56.418 INFO:teuthology.orchestra.run.smithi145.stdout:375 read 0x16c2000 thru 0x16c21dd (0x1de bytes) 2024-09-14T07:07:56.431 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:56.430+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:56.440 INFO:teuthology.orchestra.run.smithi145.stdout:376 trunc from 0x19a1600 to 0x8038400 2024-09-14T07:07:56.491 INFO:teuthology.orchestra.run.smithi145.stdout:378 write 0x9dba400 thru 0x9dc0742 (0x6343 bytes) 2024-09-14T07:07:56.494 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:56.490+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:56.496 INFO:teuthology.orchestra.run.smithi145.stdout:380 write 0x28caa00 thru 0x28d33c6 (0x89c7 bytes) 2024-09-14T07:07:56.498 INFO:teuthology.orchestra.run.smithi145.stdout:382 read 0x23a7a00 thru 0x23b5aca (0xe0cb bytes) 2024-09-14T07:07:56.500 INFO:teuthology.orchestra.run.smithi145.stdout:383 punch from 0x7702000 to 0x770ce00, (0xae00 bytes) 2024-09-14T07:07:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:56.525 INFO:teuthology.orchestra.run.smithi145.stdout:384 trunc from 0x9dc0800 to 0x883800 2024-09-14T07:07:56.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:07:56.595 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:56.594+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:56.596 INFO:teuthology.orchestra.run.smithi145.stdout:385 trunc from 0x883800 to 0xa374400 2024-09-14T07:07:56.633 INFO:teuthology.orchestra.run.smithi145.stdout:386 trunc from 0xa374400 to 0x3be6200 2024-09-14T07:07:56.671 INFO:teuthology.orchestra.run.smithi145.stdout:389 punch from 0x2af3a00 to 0x2b02600, (0xec00 bytes) 2024-09-14T07:07:56.696 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:56.694+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:56.717 INFO:teuthology.orchestra.run.smithi145.stdout:390 write 0xbb69000 thru 0xbb7043a (0x743b bytes) 2024-09-14T07:07:56.722 INFO:teuthology.orchestra.run.smithi145.stdout:392 punch from 0x4caac00 to 0x4cb7000, (0xc400 bytes) 2024-09-14T07:07:56.722 INFO:teuthology.orchestra.run.smithi145.stdout:395 punch from 0x9a54c00 to 0x9a5c600, (0x7a00 bytes) 2024-09-14T07:07:56.723 INFO:teuthology.orchestra.run.smithi145.stdout:398 read 0x4b2c400 thru 0x4b2e431 (0x2032 bytes) 2024-09-14T07:07:56.723 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:56.723 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:56.724 INFO:teuthology.orchestra.run.smithi145.stdout:399 punch from 0x2a08400 to 0x2a0ee00, (0x6a00 bytes) 2024-09-14T07:07:56.725 INFO:teuthology.orchestra.run.smithi145.stdout:400 read 0x80ec400 thru 0x80f8ee7 (0xcae8 bytes) 2024-09-14T07:07:56.725 INFO:teuthology.orchestra.run.smithi145.stdout:402 read 0x7c2d800 thru 0x7c396fe (0xbeff bytes) 2024-09-14T07:07:56.728 INFO:teuthology.orchestra.run.smithi145.stdout:404 write 0x4810800 thru 0x481566a (0x4e6b bytes) 2024-09-14T07:07:56.730 INFO:teuthology.orchestra.run.smithi145.stdout:407 write 0x3b98c00 thru 0x3ba85cc (0xf9cd bytes) 2024-09-14T07:07:56.732 INFO:teuthology.orchestra.run.smithi145.stdout:408 punch from 0xaffe000 to 0xb006600, (0x8600 bytes) 2024-09-14T07:07:56.739 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:07:56.764 INFO:teuthology.orchestra.run.smithi145.stdout:410 write 0xd2d1e00 thru 0xd2d998d (0x7b8e bytes) 2024-09-14T07:07:56.767 INFO:teuthology.orchestra.run.smithi145.stdout:411 read 0x2a7de00 thru 0x2a84040 (0x6241 bytes) 2024-09-14T07:07:56.767 INFO:teuthology.orchestra.run.smithi145.stdout:412 read 0xa190a00 thru 0xa1935e1 (0x2be2 bytes) 2024-09-14T07:07:56.768 INFO:teuthology.orchestra.run.smithi145.stdout:413 read 0x5db2200 thru 0x5db236b (0x16c bytes) 2024-09-14T07:07:56.773 INFO:teuthology.orchestra.run.smithi145.stdout:418 trunc from 0xd2d9a00 to 0xe75a800 2024-09-14T07:07:56.790 INFO:teuthology.orchestra.run.smithi145.stdout:419 write 0x2765600 thru 0x2767e9e (0x289f bytes) 2024-09-14T07:07:56.793 INFO:teuthology.orchestra.run.smithi145.stdout:420 write 0x6d23e00 thru 0x6d2da0e (0x9c0f bytes) 2024-09-14T07:07:56.795 INFO:teuthology.orchestra.run.smithi145.stdout:421 punch from 0x9619e00 to 0x9621400, (0x7600 bytes) 2024-09-14T07:07:56.796 INFO:teuthology.orchestra.run.smithi145.stdout:423 write 0x68a3e00 thru 0x68a6338 (0x2539 bytes) 2024-09-14T07:07:56.797 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:56.794+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:56.799 INFO:teuthology.orchestra.run.smithi145.stdout:424 trunc from 0xe75a800 to 0xde9d400 2024-09-14T07:07:56.819 INFO:teuthology.orchestra.run.smithi145.stdout:429 punch from 0xd1e7a00 to 0xd1e8000, (0x600 bytes) 2024-09-14T07:07:56.840 INFO:teuthology.orchestra.run.smithi145.stdout:431 trunc from 0xde9d400 to 0x70c9200 2024-09-14T07:07:56.868 INFO:teuthology.orchestra.run.smithi145.stdout:432 read 0x50e7e00 thru 0x50f0190 (0x8391 bytes) 2024-09-14T07:07:56.881 INFO:teuthology.orchestra.run.smithi145.stdout:433 trunc from 0x70c9200 to 0xa9eda00 2024-09-14T07:07:56.884 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:56.885 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:56.897 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:56.894+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:56.899 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:07:56.907 INFO:teuthology.orchestra.run.smithi145.stdout:437 read 0x2456c00 thru 0x24654a5 (0xe8a6 bytes) 2024-09-14T07:07:56.910 INFO:teuthology.orchestra.run.smithi145.stdout:441 read 0x9061a00 thru 0x906b61c (0x9c1d bytes) 2024-09-14T07:07:56.912 INFO:teuthology.orchestra.run.smithi145.stdout:442 write 0x83b2000 thru 0x83c1b54 (0xfb55 bytes) 2024-09-14T07:07:56.913 INFO:teuthology.orchestra.run.smithi145.stdout:443 write 0x1226e00 thru 0x122fe08 (0x9009 bytes) 2024-09-14T07:07:56.954 INFO:teuthology.orchestra.run.smithi145.stdout:444 write 0xcd0b200 thru 0xcd189e9 (0xd7ea bytes) 2024-09-14T07:07:56.958 INFO:teuthology.orchestra.run.smithi145.stdout:445 write 0x8328e00 thru 0x833825e (0xf45f bytes) 2024-09-14T07:07:56.959 INFO:teuthology.orchestra.run.smithi145.stdout:448 write 0x2e1b000 thru 0x2e26306 (0xb307 bytes) 2024-09-14T07:07:56.961 INFO:teuthology.orchestra.run.smithi145.stdout:451 write 0x100ba00 thru 0x1018a81 (0xd082 bytes) 2024-09-14T07:07:56.963 INFO:teuthology.orchestra.run.smithi145.stdout:454 write 0x1842200 thru 0x184329e (0x109f bytes) 2024-09-14T07:07:56.965 INFO:teuthology.orchestra.run.smithi145.stdout:455 clone 2 order 18 su 131072 sc 6 2024-09-14T07:07:56.998 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:56.994+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:57.068 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:57.068 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:57.081 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:07:57.099 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:57.098+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:57.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:07:57.200 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:57.198+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:57.204 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:57.204 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:57.221 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:07:57.270 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:07:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:07:57.287 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:07:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:07:57.304 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:07:57.304 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:07:57.311 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:57.310+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:57.321 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:07:57.322 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:07:57.341 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:07:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:07:57.361 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:07:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:07:57.391 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:57.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:57.391 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:07:57.391 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:57.401 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:57.398+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:57.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:07:57.502 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:57.498+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:57.571 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:07:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:07:57.571 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:57.585 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:07:57.603 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:57.602+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:57.703 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:57.702+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.9568817329999995, 2024-09-14T07:07:57.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070947240000000002, 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 7 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793553+0000", 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3730000000000001e-06 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793583+0000", 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9675999999999998e-05 2024-09-14T07:07:57.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793621+0000", 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7370000000000003e-05 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793647+0000", 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6276999999999999e-05 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793835+0000", 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018812500000000001 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793852+0000", 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.734e-05 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797434+0000", 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035819189999999998 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797559+0000", 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012508500000000001 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797726+0000", 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000166797 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.798685+0000", 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00095928199999999999 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800546+0000", 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018609189999999999 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800602+0000", 2024-09-14T07:07:57.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5365999999999998e-05 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800645+0000", 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3195000000000002e-05 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-14T07:07:57.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.949448222, 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0097102179999999996, 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800987+0000", 2024-09-14T07:07:57.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801000+0000", 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2897e-05 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801028+0000", 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8575999999999999e-05 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801054+0000", 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6367e-05 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801282+0000", 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022778100000000001 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801300+0000", 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7237000000000001e-05 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808053+0000", 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0067535249999999998 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808316+0000", 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000263224 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808688+0000", 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037128699999999998 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810367+0000", 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001679151 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810427+0000", 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0338999999999999e-05 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810572+0000", 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014462200000000001 2024-09-14T07:07:57.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810694+0000", 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000122192 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:929 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:07:57.766 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.91086147, 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013302158999999999, 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 929 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:07:57.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839572+0000", 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6780000000000001e-06 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839580+0000", 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0989999999999999e-06 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839590+0000", 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839598+0000", 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3699999999999997e-06 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840130+0000", 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00053217 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840818+0000", 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00068814799999999997 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845641+0000", 2024-09-14T07:07:57.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048226160000000001 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845730+0000", 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9687999999999999e-05 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846136+0000", 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000405583 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849695+0000", 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00355887 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852725+0000", 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030301009999999999 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852786+0000", 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1081999999999996e-05 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852873+0000", 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6608999999999997e-05 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:930 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9108218880000001, 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013501599, 2024-09-14T07:07:57.773 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 930 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839613+0000", 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4169999999999999e-06 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839618+0000", 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1499999999999998e-06 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839622+0000", 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.631e-06 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841189+0000", 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001567768 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841262+0000", 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2105000000000002e-05 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846495+0000", 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052335999999999997 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846559+0000", 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3609999999999996e-05 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846782+0000", 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022373299999999999 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849705+0000", 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002922998 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852940+0000", 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032343070000000001 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853014+0000", 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3967999999999994e-05 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853112+0000", 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7806000000000005e-05 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:931 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.779 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.910753583, 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.01360341, 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 931 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839679+0000", 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7299999999999996e-07 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839682+0000", 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839688+0000", 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9569999999999999e-06 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839693+0000", 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6289999999999998e-06 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841324+0000", 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016306090000000001 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841425+0000", 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010129299999999999 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847149+0000", 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057235139999999999 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847194+0000", 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5117000000000002e-05 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847441+0000", 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000247428 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852669+0000", 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052276029999999999 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853162+0000", 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049292300000000004 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853202+0000", 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0525999999999998e-05 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853282+0000", 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9513999999999998e-05 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.785 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:932 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9106467609999997, 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013718054, 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 932 2024-09-14T07:07:57.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839786+0000", 2024-09-14T07:07:57.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4499999999999996e-07 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839794+0000", 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6450000000000005e-06 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839805+0000", 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1489e-05 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839814+0000", 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6540000000000005e-06 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841579+0000", 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017656480000000001 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841669+0000", 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9806e-05 2024-09-14T07:07:57.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847963+0000", 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0062937080000000003 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848135+0000", 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017191299999999999 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848791+0000", 2024-09-14T07:07:57.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00065584300000000003 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853128+0000", 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043374149999999998 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853351+0000", 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000223408 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853408+0000", 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6922000000000002e-05 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853503+0000", 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4758000000000003e-05 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:933 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:07:57.792 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9102158070000002, 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015741667000000001, 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 933 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:07:57.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840217+0000", 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1909999999999999e-06 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840230+0000", 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3226e-05 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840244+0000", 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3108999999999999e-05 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840252+0000", 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2910000000000007e-06 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841706+0000", 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001454056 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841766+0000", 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0494000000000001e-05 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849824+0000", 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080573079999999991 2024-09-14T07:07:57.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850028+0000", 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020432999999999999 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850748+0000", 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072041399999999995 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853135+0000", 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002386885 2024-09-14T07:07:57.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855783+0000", 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026479350000000001 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855851+0000", 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7534999999999996e-05 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855958+0000", 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010689299999999999 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.798 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:935 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9085066689999999, 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016747201, 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 935 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841926+0000", 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841932+0000", 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2099999999999998e-06 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841943+0000", 2024-09-14T07:07:57.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0981e-05 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841950+0000", 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842004+0000", 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4422000000000002e-05 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:57.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842064+0000", 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9967999999999999e-05 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851737+0000", 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0096733600000000006 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851825+0000", 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7855999999999994e-05 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852258+0000", 2024-09-14T07:07:57.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000432383 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853520+0000", 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001262787 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858562+0000", 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050418269999999996 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858623+0000", 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0789999999999999e-05 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858672+0000", 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9348999999999998e-05 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:936 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.805 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9082649109999998, 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.018202472000000001, 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 936 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:07:57.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842168+0000", 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3300000000000002e-07 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842170+0000", 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6539999999999999e-06 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842178+0000", 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.413e-06 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842182+0000", 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3139999999999997e-06 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842231+0000", 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8553000000000001e-05 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842290+0000", 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9534000000000003e-05 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853603+0000", 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011312579 2024-09-14T07:07:57.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853796+0000", 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299000000000001 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854842+0000", 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00104612 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858690+0000", 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038478589999999999 2024-09-14T07:07:57.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860251+0000", 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015616829999999999 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860315+0000", 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3587e-05 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860369+0000", 2024-09-14T07:07:57.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4552999999999998e-05 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:937 3.11 e26/21 3:8ce9c1c4:::rbd_data.11611b5ca054.0000000000000026:head v 26'3, mlcod=24'1)", 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.853952+0000", 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8964795190000001, 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047661400000000003, 2024-09-14T07:07:57.812 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853952+0000", 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853952+0000", 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853954+0000", 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853987+0000", 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3602000000000001e-05 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854007+0000", 2024-09-14T07:07:57.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9806e-05 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854021+0000", 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3757999999999999e-05 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855996+0000", 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019747599999999999 2024-09-14T07:07:57.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856019+0000", 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.298e-05 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858693+0000", 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026738249999999999 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858718+0000", 2024-09-14T07:07:57.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950999999999999e-05 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1141 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.016926+0000", 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7335057320000002, 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006443926, 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1141 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016926+0000", 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016926+0000", 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016928+0000", 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7039999999999999e-06 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016937+0000", 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-06 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016961+0000", 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4241000000000001e-05 2024-09-14T07:07:57.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.016994+0000", 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2335999999999997e-05 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017103+0000", 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109522 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017308+0000", 2024-09-14T07:07:57.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020519900000000001 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.018237+0000", 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00092822100000000002 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023056+0000", 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048193039999999999 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023174+0000", 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118393 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023271+0000", 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6446000000000005e-05 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023370+0000", 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9291999999999997e-05 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.822 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1142 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7333867129999998, 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.007319173, 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1142 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017046+0000", 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017061+0000", 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.467e-05 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017076+0000", 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5421e-05 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017088+0000", 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1287999999999999e-05 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019464+0000", 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023761590000000001 2024-09-14T07:07:57.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019645+0000", 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000180772 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.020444+0000", 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00079905700000000004 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023077+0000", 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002633093 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024178+0000", 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011013850000000001 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024296+0000", 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117586 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024364+0000", 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8585999999999997e-05 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1145 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7330883930000001, 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077736239999999998, 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.829 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1145 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017346+0000", 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2369999999999999e-06 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017358+0000", 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2602999999999999e-05 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017390+0000", 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.188e-05 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017413+0000", 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.243e-05 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021308+0000", 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038951799999999998 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021511+0000", 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020368099999999999 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.022262+0000", 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00075023799999999997 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023822+0000", 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015606750000000001 2024-09-14T07:07:57.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025018+0000", 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001195149 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8764999999999997e-05 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025117+0000", 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0785999999999999e-05 2024-09-14T07:07:57.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1159 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7254629690000001, 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0074537930000000002, 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1159 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024970+0000", 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5939999999999999e-06 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024981+0000", 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025004+0000", 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3468999999999998e-05 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025021+0000", 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6674e-05 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025194+0000", 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173721 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025445+0000", 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025037099999999999 2024-09-14T07:07:57.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.026330+0000", 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000885073 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032132+0000", 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058019819999999998 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032250+0000", 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118447 2024-09-14T07:07:57.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032351+0000", 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100966 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032423+0000", 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1351000000000002e-05 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1160 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7253570360000001, 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0076908269999999999, 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1160 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.841 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.251e-06 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025084+0000", 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5689999999999999e-06 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025102+0000", 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8734000000000001e-05 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025116+0000", 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3465999999999999e-05 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027183+0000", 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002066877 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027355+0000", 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017235800000000001 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028074+0000", 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00071921500000000005 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032586+0000", 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045120960000000002 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032617+0000", 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0468999999999999e-05 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032697+0000", 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9939999999999997e-05 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032766+0000", 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8851999999999997e-05 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1161 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.846 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7252588310000001, 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0078300520000000005, 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1161 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:07:57.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025174+0000", 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4139999999999999e-06 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025184+0000", 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3489999999999998e-06 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025201+0000", 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7011e-05 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025214+0000", 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3237e-05 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028886+0000", 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036715649999999999 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029039+0000", 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015298300000000001 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029611+0000", 2024-09-14T07:07:57.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057197599999999997 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032782+0000", 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031712340000000002 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032863+0000", 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0907000000000003e-05 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032957+0000", 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4715000000000004e-05 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033003+0000", 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5661000000000002e-05 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1162 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.852 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7251911689999999, 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084900349999999999, 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1162 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025242+0000", 2024-09-14T07:07:57.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3829999999999999e-06 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025248+0000", 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1999999999999999e-06 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025283+0000", 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4347000000000002e-05 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025295+0000", 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2268e-05 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030146+0000", 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048513200000000001 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030309+0000", 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000163225 2024-09-14T07:07:57.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.031281+0000", 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00097148600000000003 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033053+0000", 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001771941 2024-09-14T07:07:57.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033582+0000", 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052929400000000003 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033653+0000", 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1285999999999995e-05 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033731+0000", 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7285000000000003e-05 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1167 3.a e26/21 3:509716c6:::rbd_data.11611b5ca054.000000000000002a:head v 26'12, mlcod=26'10)", 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7163185790000002, 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065049039999999997, 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.859 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034114+0000", 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0800000000000003e-07 2024-09-14T07:07:57.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034143+0000", 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8532000000000001e-05 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034158+0000", 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5525000000000001e-05 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034169+0000", 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0726e-05 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034296+0000", 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012745700000000001 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034352+0000", 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5229999999999998e-05 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040532+0000", 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0061802130000000004 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040618+0000", 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6312999999999998e-05 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.863 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1586 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.348680+0000", 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.401751371, 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042129139999999999, 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1586 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348680+0000", 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348680+0000", 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348681+0000", 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4700000000000001e-07 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348686+0000", 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3900000000000003e-06 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348700+0000", 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4388e-05 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348707+0000", 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1679999999999998e-06 2024-09-14T07:07:57.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.349668+0000", 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096081900000000002 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.349851+0000", 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018299200000000001 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.350291+0000", 2024-09-14T07:07:57.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044031700000000002 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.351782+0000", 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014911620000000001 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352786+0000", 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010035409999999999 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352850+0000", 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3953000000000002e-05 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352893+0000", 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3436999999999998e-05 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1587 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.348720+0000", 2024-09-14T07:07:57.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.4017119830000002, 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051567840000000002, 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1587 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348720+0000", 2024-09-14T07:07:57.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348720+0000", 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348721+0000", 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7299999999999995e-07 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348727+0000", 2024-09-14T07:07:57.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0399999999999998e-06 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348735+0000", 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5299999999999996e-06 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.348742+0000", 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6769999999999998e-06 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:57.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.350868+0000", 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021257490000000001 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.350986+0000", 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011850599999999999 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.351385+0000", 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00039872699999999998 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.352424+0000", 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010396590000000001 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.353783+0000", 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013586049999999999 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.353836+0000", 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2787999999999997e-05 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:57.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.353877+0000", 2024-09-14T07:07:57.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.083e-05 2024-09-14T07:07:57.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:57.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:57.875 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:57.875 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:07:57.876 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:07:57.876 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T07:07:57.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:07:57.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:07:57.878 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:07:57.879 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:57.874+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:57.978 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:57.974+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:58.052 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:07:58.052 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:58.067 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:07:58.115 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:58.114+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:58.180 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:58.178+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:58.201 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:58.201 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:58.201 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 29, 2024-09-14T07:07:58.201 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:07:58.201 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:07:58.201 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:07:57.746134+0000", 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:07:52.806531+0000", 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5799936, 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:07:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:07:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:07:58.204 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:07:58.205 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:07:58.206 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:07:58.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:07:58.208 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:07:58.209 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:07:58.210 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 6.059999942779541, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 6.059999942779541, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.33000001311302185, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:07:58.211 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:07:58.212 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:07:58.213 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:07:58.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:07:58.215 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:07:58.216 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:07:58.217 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 2.25, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2.25, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:07:58.218 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:07:58.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:07:58.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:07:58.221 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "29", 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 4, 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 29, 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:07:58.223 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:07:58.224 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.5, 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.5, 2024-09-14T07:07:58.225 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.5, 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.5, 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:07:58.226 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 21, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.228 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:07:58.229 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:07:58.232 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:07:58.233 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.234 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.236 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.237 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:07:58.238 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:07:58.239 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:07:58.241 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:07:58.241 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:07:58.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.241 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:07:58.241 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:07:58.242 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.243 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:07:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.246 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:07:58.247 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:07:58.248 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:07:58.249 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.252 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:07:58.253 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:07:58.254 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:07:58.255 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:07:58.257 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.258 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:07:58.259 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:07:58.261 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:07:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:07:58.266 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:07:58.267 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:07:58.268 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:07:58.269 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:07:58.270 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-09-14T07:07:58.272 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:07:58.273 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [ 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: "snaps": [ 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: "begin": 2, 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: "length": 1 2024-09-14T07:07:58.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:07:58.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:58.276 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:58.277 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:58.277 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:07:58.278 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:07:58.278 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:07:58.278 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:58.278 INFO:tasks.thrashosds.thrasher:j is {'epoch': 29, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:07:57.746134+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:07:52.806531+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 29, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:07:58.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:07:58.279 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:07:58.281 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:58.278+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:58.382 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:58.382+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:58.395 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T07:07:58.447 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:07:58.447 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4199.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:37.024310+0000", 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.421978081999999, 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064646900000000004, 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4199", 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.5:0/4051694687", 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 5 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024310+0000", 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024310+0000", 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024313+0000", 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9000000000000002e-06 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024334+0000", 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0743000000000001e-05 2024-09-14T07:07:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024367+0000", 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3683000000000003e-05 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024385+0000", 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7413000000000001e-05 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024514+0000", 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012901899999999999 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.024525+0000", 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0837e-05 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.027793+0000", 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032689339999999998 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.027986+0000", 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000192222 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.028179+0000", 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019309899999999999 2024-09-14T07:07:58.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.029590+0000", 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014117229999999999 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030498+0000", 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000907266 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030649+0000", 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015092600000000001 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:37.030775+0000", 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012592500000000001 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4199.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:38.032884+0000", 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 20.413404286999999, 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066585289999999998, 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4199", 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.5:0/4051694687", 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 7 2024-09-14T07:07:58.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032884+0000", 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032884+0000", 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032887+0000", 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.444e-06 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032902+0000", 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4430000000000001e-05 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032937+0000", 2024-09-14T07:07:58.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5366e-05 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.032963+0000", 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.654e-05 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.033116+0000", 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015216900000000001 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.033135+0000", 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8960000000000001e-05 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.036494+0000", 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033589869999999999 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.036625+0000", 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131159 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.036837+0000", 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000211759 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.038340+0000", 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001503211 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039391+0000", 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001051202 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039480+0000", 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9069999999999999e-05 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:38.039542+0000", 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2231999999999996e-05 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:49.812465+0000", 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.6338232579999996, 2024-09-14T07:07:58.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0096818990000000008, 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812465+0000", 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812465+0000", 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812468+0000", 2024-09-14T07:07:58.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812481+0000", 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2869e-05 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812513+0000", 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2453000000000003e-05 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812536+0000", 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2874000000000002e-05 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812711+0000", 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000174599 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.812723+0000", 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2499e-05 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.817652+0000", 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0049287589999999996 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.818027+0000", 2024-09-14T07:07:58.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037505699999999998 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.818580+0000", 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00055272299999999997 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.820943+0000", 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023637409999999999 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.821896+0000", 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00095269599999999995 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.822060+0000", 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000163826 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.822147+0000", 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6706000000000006e-05 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:379 3.1d 3:b8c07fa4:::rbd_header.11611b5ca054:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.814369+0000", 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.631919356, 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010161081000000001, 2024-09-14T07:07:58.471 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 379 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814369+0000", 2024-09-14T07:07:58.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814369+0000", 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814370+0000", 2024-09-14T07:07:58.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8369999999999999e-06 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814381+0000", 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0219e-05 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814411+0000", 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.029e-05 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814426+0000", 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814751+0000", 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032523099999999999 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814770+0000", 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8425999999999999e-05 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.823545+0000", 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00877556 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.823750+0000", 2024-09-14T07:07:58.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020448999999999999 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.824530+0000", 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00078010799999999995 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.477 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:919 3.1d 3:b8c07fa4:::rbd_header.11611b5ca054:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.815581+0000", 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6307070689999996, 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010542739000000001, 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 919 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.478 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815581+0000", 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815581+0000", 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815583+0000", 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815595+0000", 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1759999999999999e-05 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815633+0000", 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8503e-05 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815656+0000", 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2507e-05 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815820+0000", 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000164291 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815837+0000", 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6864e-05 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.824604+0000", 2024-09-14T07:07:58.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0087665460000000001 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.824811+0000", 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020722399999999999 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.826124+0000", 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001312864 2024-09-14T07:07:58.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6157509069999998, 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.484 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:07:58.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:07:58.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:07:58.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:07:58.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.490 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6149765590000005, 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.491 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:07:58.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:07:58.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.496 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:928 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [write 933888~49152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839499+0000", 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.6067889109999998, 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017193320000000002, 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 928 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839499+0000", 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839499+0000", 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839500+0000", 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839653+0000", 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015312100000000001 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839672+0000", 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8629000000000001e-05 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839681+0000", 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8759999999999994e-06 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839746+0000", 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5134000000000004e-05 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840343+0000", 2024-09-14T07:07:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00059738 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845463+0000", 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051200509999999996 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845520+0000", 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6552999999999999e-05 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845627+0000", 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106894 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849612+0000", 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039852749999999999 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856485+0000", 2024-09-14T07:07:58.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0068730379999999997 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856567+0000", 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2553999999999996e-05 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856692+0000", 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012496299999999999 2024-09-14T07:07:58.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:940 3.1d 3:bc582f18:::rbd_data.11611b5ca054.0000000000000033:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4009984~41472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.853670+0000", 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5926176249999999, 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0055221549999999999, 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 940 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853670+0000", 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853670+0000", 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853672+0000", 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8700000000000001e-06 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853722+0000", 2024-09-14T07:07:58.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0123000000000001e-05 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853745+0000", 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2778000000000001e-05 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853756+0000", 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0548e-05 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853849+0000", 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3246999999999997e-05 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854010+0000", 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016138200000000001 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854183+0000", 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173027 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855587+0000", 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014036750000000001 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.859058+0000", 2024-09-14T07:07:58.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034713719999999999 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.859135+0000", 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6571999999999998e-05 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.859193+0000", 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7561e-05 2024-09-14T07:07:58.511 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:939 3.2 e26/21 3:40fc327c:::rbd_data.11611b5ca054.000000000000002e:head v 26'7, mlcod=24'5)", 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.855132+0000", 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5911557949999997, 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0049341899999999998, 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.512 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855132+0000", 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855132+0000", 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855134+0000", 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5650000000000001e-06 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855339+0000", 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000204916 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855362+0000", 2024-09-14T07:07:58.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3685000000000001e-05 2024-09-14T07:07:58.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855377+0000", 2024-09-14T07:07:58.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5082999999999999e-05 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855497+0000", 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119713 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855525+0000", 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8160000000000001e-05 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860036+0000", 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045103729999999998 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860066+0000", 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0694999999999997e-05 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1014 3.1d 3:bc582f18:::rbd_data.11611b5ca054.0000000000000033:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.934119+0000", 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5121690379999997, 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003596995, 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1014 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934119+0000", 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934119+0000", 2024-09-14T07:07:58.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934120+0000", 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.477e-06 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934126+0000", 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.071e-06 2024-09-14T07:07:58.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934146+0000", 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9502999999999999e-05 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934161+0000", 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4826e-05 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934657+0000", 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049603099999999997 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934726+0000", 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8891000000000002e-05 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.934820+0000", 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3889999999999997e-05 2024-09-14T07:07:58.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.937399+0000", 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002579731 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.937583+0000", 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018315900000000001 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.937657+0000", 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4852999999999996e-05 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.937716+0000", 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8563000000000002e-05 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1155 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 8192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024544+0000", 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4217438290000004, 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043032510000000001, 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1155 2024-09-14T07:07:58.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024544+0000", 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024544+0000", 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024545+0000", 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1000000000000001e-06 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024555+0000", 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0163000000000001e-05 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024594+0000", 2024-09-14T07:07:58.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8448000000000002e-05 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024603+0000", 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.329e-06 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028348+0000", 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003744355 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028563+0000", 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021560700000000001 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028847+0000", 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028424899999999998 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1156 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 32768~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024623+0000", 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4216653810000004, 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046521660000000001, 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1156 2024-09-14T07:07:58.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024623+0000", 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024623+0000", 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024624+0000", 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3659999999999999e-06 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024630+0000", 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9739999999999999e-06 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024649+0000", 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024660+0000", 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0397000000000001e-05 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028935+0000", 2024-09-14T07:07:58.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042751669999999999 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029078+0000", 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014263700000000001 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029275+0000", 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000197262 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.534 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1157 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [sparse-read 65536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024672+0000", 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4216163980000003, 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051540839999999997, 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1157 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024672+0000", 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024672+0000", 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024672+0000", 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1399999999999996e-07 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024675+0000", 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8720000000000002e-06 2024-09-14T07:07:58.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024684+0000", 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4190000000000002e-06 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024690+0000", 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9150000000000001e-06 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029340+0000", 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046506509999999996 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029546+0000", 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020554899999999999 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029826+0000", 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000279864 2024-09-14T07:07:58.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1168 3.2 e26/21 3:42220d69:::rbd_data.11611b5ca054.000000000000002d:head v 26'9, mlcod=26'7)", 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.034190+0000", 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4120984009999997, 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013133113, 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.540 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034190+0000", 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034190+0000", 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034192+0000", 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.984e-06 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034237+0000", 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4963000000000003e-05 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034273+0000", 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6423e-05 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034293+0000", 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9686000000000001e-05 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037078+0000", 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027857659999999998 2024-09-14T07:07:58.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037132+0000", 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3535999999999997e-05 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047259+0000", 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010127038 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047323+0000", 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3717e-05 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1170 3.2 e26/21 3:40d5ec5b:::rbd_data.11611b5ca054.000000000000002f:head v 26'10, mlcod=26'7)", 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.037387+0000", 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4089007499999999, 2024-09-14T07:07:58.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010559537000000001, 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037387+0000", 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037387+0000", 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037389+0000", 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.68e-06 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037489+0000", 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100574 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037517+0000", 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7185000000000001e-05 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037553+0000", 2024-09-14T07:07:58.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6714000000000001e-05 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040335+0000", 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027815140000000001 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040380+0000", 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4965000000000001e-05 2024-09-14T07:07:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047842+0000", 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0074618490000000004 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047947+0000", 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010505600000000001 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1195 3.1d 3:bc582f18:::rbd_data.11611b5ca054.0000000000000033:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.057895+0000", 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3883933219999998, 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034684220000000001, 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1195 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.057895+0000", 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.057895+0000", 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.057896+0000", 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.057908+0000", 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2046e-05 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.057927+0000", 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9119000000000001e-05 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.057938+0000", 2024-09-14T07:07:58.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1056999999999999e-05 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.058022+0000", 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3948000000000003e-05 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.058107+0000", 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4583e-05 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.058234+0000", 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012740900000000001 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.060153+0000", 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001918899 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.061270+0000", 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001116403 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.061319+0000", 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9573999999999997e-05 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.061363+0000", 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3954e-05 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1647 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 131072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.382073+0000", 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0642149490000001, 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033004929999999998, 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.558 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1647 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382073+0000", 2024-09-14T07:07:58.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382073+0000", 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382074+0000", 2024-09-14T07:07:58.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3300000000000001e-07 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382077+0000", 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7999999999999999e-06 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382085+0000", 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7439999999999996e-06 2024-09-14T07:07:58.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382092+0000", 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.384937+0000", 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028447519999999999 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.385130+0000", 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000193315 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.385374+0000", 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024345599999999999 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1648 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 139264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.382101+0000", 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0641865890000002, 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035400750000000002, 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1648 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382101+0000", 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382101+0000", 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382102+0000", 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382105+0000", 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.88e-06 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382114+0000", 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6400000000000003e-06 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.382121+0000", 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9500000000000004e-06 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.385442+0000", 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033210760000000001 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.385535+0000", 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3479000000000004e-05 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.385641+0000", 2024-09-14T07:07:58.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106327 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1761 3.1d 3:b8c07fa4:::rbd_header.11611b5ca054:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.827964+0000", 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.6183243269999998, 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0069906120000000002, 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:07:58.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1761 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.827964+0000", 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.827964+0000", 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.827967+0000", 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2140000000000001e-06 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:07:58.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.827982+0000", 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-05 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.828021+0000", 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8559000000000001e-05 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.828045+0000", 2024-09-14T07:07:58.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4409999999999998e-05 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.828162+0000", 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116568 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.828170+0000", 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0239999999999997e-06 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.831822+0000", 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00365227 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.831929+0000", 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010708 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.832350+0000", 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000421159 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.834114+0000", 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001763602 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:58.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.834679+0000", 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056516900000000002 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.834851+0000", 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017147500000000001 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.834954+0000", 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010359199999999999 2024-09-14T07:07:58.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:07:58.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:07:58.577 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:07:58.579 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:07:58.574+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:07:58.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:07:58.605 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:07:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:07:58.627 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-14T07:07:57.748817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35831022,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":5786,"num_read_kb":8345,"num_write":554,"num_write_kb":4581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":139311,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":522,"ondisk_log_size":522,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":197832,"kb_used_data":2368,"kb_used_omap":9,"kb_used_meta":195318,"kb_avail":566033208,"statfs":{"total":579820584960,"available":579618004992,"internally_reserved":0,"allocated":2424832,"data_stored":3397421,"data_compressed":349097,"data_compressed_allocated":536576,"data_compressed_original":2707456,"omap_allocated":9568,"internal_metadata":200006304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34589803,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5422,"num_read_kb":8045,"num_write":381,"num_write_kb":2841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":139311,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.539831"},"pg_stats":[{"pgid":"3.1f","version":"24'1","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.828623+0000","last_change":"2024-09-14T07:07:49.800921+0000","last_active":"2024-09-14T07:07:54.828623+0000","last_peered":"2024-09-14T07:07:54.828623+0000","last_clean":"2024-09-14T07:07:54.828623+0000","last_became_active":"2024-09-14T07:07:47.794275+0000","last_became_peered":"2024-09-14T07:07:47.794275+0000","last_unstale":"2024-09-14T07:07:54.828623+0000","last_undegraded":"2024-09-14T07:07:54.828623+0000","last_fullsized":"2024-09-14T07:07:54.828623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:46.276063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"26'5","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:55.150264+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:07:55.150264+0000","last_peered":"2024-09-14T07:07:55.150264+0000","last_clean":"2024-09-14T07:07:55.150264+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:07:55.150264+0000","last_undegraded":"2024-09-14T07:07:55.150264+0000","last_fullsized":"2024-09-14T07:07:55.150264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:30:21.601228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":503808,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":27,"num_read_kb":348,"num_write":5,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"27'107","reported_seq":1394,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:55.218454+0000","last_change":"2024-09-14T07:07:49.819484+0000","last_active":"2024-09-14T07:07:55.218454+0000","last_peered":"2024-09-14T07:07:55.218454+0000","last_clean":"2024-09-14T07:07:55.218454+0000","last_became_active":"2024-09-14T07:07:47.787620+0000","last_became_peered":"2024-09-14T07:07:47.787620+0000","last_unstale":"2024-09-14T07:07:55.218454+0000","last_undegraded":"2024-09-14T07:07:55.218454+0000","last_fullsized":"2024-09-14T07:07:55.218454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:09:08.543424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2523,"num_read_kb":2404,"num_write":55,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.895377+0000","last_change":"2024-09-14T07:07:49.820020+0000","last_active":"2024-09-14T07:07:54.895377+0000","last_peered":"2024-09-14T07:07:54.895377+0000","last_clean":"2024-09-14T07:07:54.895377+0000","last_became_active":"2024-09-14T07:07:47.797935+0000","last_became_peered":"2024-09-14T07:07:47.797935+0000","last_unstale":"2024-09-14T07:07:54.895377+0000","last_undegraded":"2024-09-14T07:07:54.895377+0000","last_fullsized":"2024-09-14T07:07:54.895377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:54:48.281185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"24'4","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.828469+0000","last_change":"2024-09-14T07:07:49.801349+0000","last_active":"2024-09-14T07:07:54.828469+0000","last_peered":"2024-09-14T07:07:54.828469+0000","last_clean":"2024-09-14T07:07:54.828469+0000","last_became_active":"2024-09-14T07:07:47.789185+0000","last_became_peered":"2024-09-14T07:07:47.789185+0000","last_unstale":"2024-09-14T07:07:54.828469+0000","last_undegraded":"2024-09-14T07:07:54.828469+0000","last_fullsized":"2024-09-14T07:07:54.828469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:20.101490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"28'76","reported_seq":1082,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T07:07:56.992924+0000","last_change":"2024-09-14T07:07:49.810147+0000","last_active":"2024-09-14T07:07:56.992924+0000","last_peered":"2024-09-14T07:07:56.992924+0000","last_clean":"2024-09-14T07:07:56.992924+0000","last_became_active":"2024-09-14T07:07:47.794385+0000","last_became_peered":"2024-09-14T07:07:47.794385+0000","last_unstale":"2024-09-14T07:07:56.992924+0000","last_undegraded":"2024-09-14T07:07:56.992924+0000","last_fullsized":"2024-09-14T07:07:56.992924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:53:18.436468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1871,"num_read_kb":1643,"num_write":40,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"27'2","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.910082+0000","last_change":"2024-09-14T07:07:49.812943+0000","last_active":"2024-09-14T07:07:54.910082+0000","last_peered":"2024-09-14T07:07:54.910082+0000","last_clean":"2024-09-14T07:07:54.910082+0000","last_became_active":"2024-09-14T07:07:47.792709+0000","last_became_peered":"2024-09-14T07:07:47.792709+0000","last_unstale":"2024-09-14T07:07:54.910082+0000","last_undegraded":"2024-09-14T07:07:54.910082+0000","last_fullsized":"2024-09-14T07:07:54.910082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:13:46.193140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":2576422,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":148,"num_write":2,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"26'5","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.845813+0000","last_change":"2024-09-14T07:07:53.859978+0000","last_active":"2024-09-14T07:07:54.845813+0000","last_peered":"2024-09-14T07:07:54.845813+0000","last_clean":"2024-09-14T07:07:54.845813+0000","last_became_active":"2024-09-14T07:07:53.840766+0000","last_became_peered":"2024-09-14T07:07:53.840766+0000","last_unstale":"2024-09-14T07:07:54.845813+0000","last_undegraded":"2024-09-14T07:07:54.845813+0000","last_fullsized":"2024-09-14T07:07:54.845813+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:02:49.104853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":8192,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.829089+0000","last_change":"2024-09-14T07:07:49.801370+0000","last_active":"2024-09-14T07:07:54.829089+0000","last_peered":"2024-09-14T07:07:54.829089+0000","last_clean":"2024-09-14T07:07:54.829089+0000","last_became_active":"2024-09-14T07:07:47.780206+0000","last_became_peered":"2024-09-14T07:07:47.780206+0000","last_unstale":"2024-09-14T07:07:54.829089+0000","last_undegraded":"2024-09-14T07:07:54.829089+0000","last_fullsized":"2024-09-14T07:07:54.829089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:22:18.078133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.833247+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:07:54.833247+0000","last_peered":"2024-09-14T07:07:54.833247+0000","last_clean":"2024-09-14T07:07:54.833247+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:07:54.833247+0000","last_undegraded":"2024-09-14T07:07:54.833247+0000","last_fullsized":"2024-09-14T07:07:54.833247+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:23.200475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"27'5","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.891094+0000","last_change":"2024-09-14T07:07:49.804465+0000","last_active":"2024-09-14T07:07:54.891094+0000","last_peered":"2024-09-14T07:07:54.891094+0000","last_clean":"2024-09-14T07:07:54.891094+0000","last_became_active":"2024-09-14T07:07:47.804440+0000","last_became_peered":"2024-09-14T07:07:47.804440+0000","last_unstale":"2024-09-14T07:07:54.891094+0000","last_undegraded":"2024-09-14T07:07:54.891094+0000","last_fullsized":"2024-09-14T07:07:54.891094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:21:20.450552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":3223601,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":3,"num_write":5,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.826514+0000","last_change":"2024-09-14T07:07:49.803381+0000","last_active":"2024-09-14T07:07:54.826514+0000","last_peered":"2024-09-14T07:07:54.826514+0000","last_clean":"2024-09-14T07:07:54.826514+0000","last_became_active":"2024-09-14T07:07:47.794784+0000","last_became_peered":"2024-09-14T07:07:47.794784+0000","last_unstale":"2024-09-14T07:07:54.826514+0000","last_undegraded":"2024-09-14T07:07:54.826514+0000","last_fullsized":"2024-09-14T07:07:54.826514+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:55:17.537028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.827157+0000","last_change":"2024-09-14T07:07:38.031294+0000","last_active":"2024-09-14T07:07:54.827157+0000","last_peered":"2024-09-14T07:07:54.827157+0000","last_clean":"2024-09-14T07:07:54.827157+0000","last_became_active":"2024-09-14T07:07:36.418930+0000","last_became_peered":"2024-09-14T07:07:36.418930+0000","last_unstale":"2024-09-14T07:07:54.827157+0000","last_undegraded":"2024-09-14T07:07:54.827157+0000","last_fullsized":"2024-09-14T07:07:54.827157+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:08.172090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'4","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.828683+0000","last_change":"2024-09-14T07:07:49.801738+0000","last_active":"2024-09-14T07:07:54.828683+0000","last_peered":"2024-09-14T07:07:54.828683+0000","last_clean":"2024-09-14T07:07:54.828683+0000","last_became_active":"2024-09-14T07:07:47.793157+0000","last_became_peered":"2024-09-14T07:07:47.793157+0000","last_unstale":"2024-09-14T07:07:54.828683+0000","last_undegraded":"2024-09-14T07:07:54.828683+0000","last_fullsized":"2024-09-14T07:07:54.828683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:36:29.501082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":36,"num_write":4,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.900840+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:07:54.900840+0000","last_peered":"2024-09-14T07:07:54.900840+0000","last_clean":"2024-09-14T07:07:54.900840+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:07:54.900840+0000","last_undegraded":"2024-09-14T07:07:54.900840+0000","last_fullsized":"2024-09-14T07:07:54.900840+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"26'5","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.845780+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:07:54.845780+0000","last_peered":"2024-09-14T07:07:54.845780+0000","last_clean":"2024-09-14T07:07:54.845780+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:07:54.845780+0000","last_undegraded":"2024-09-14T07:07:54.845780+0000","last_fullsized":"2024-09-14T07:07:54.845780+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:41:54.424658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":27,"num_write":5,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.827223+0000","last_change":"2024-09-14T07:07:38.031148+0000","last_active":"2024-09-14T07:07:54.827223+0000","last_peered":"2024-09-14T07:07:54.827223+0000","last_clean":"2024-09-14T07:07:54.827223+0000","last_became_active":"2024-09-14T07:07:36.196549+0000","last_became_peered":"2024-09-14T07:07:36.196549+0000","last_unstale":"2024-09-14T07:07:54.827223+0000","last_undegraded":"2024-09-14T07:07:54.827223+0000","last_fullsized":"2024-09-14T07:07:54.827223+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:35.911258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"24'5","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.900787+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:07:54.900787+0000","last_peered":"2024-09-14T07:07:54.900787+0000","last_clean":"2024-09-14T07:07:54.900787+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:07:54.900787+0000","last_undegraded":"2024-09-14T07:07:54.900787+0000","last_fullsized":"2024-09-14T07:07:54.900787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:35.513585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.900859+0000","last_change":"2024-09-14T07:07:38.032270+0000","last_active":"2024-09-14T07:07:54.900859+0000","last_peered":"2024-09-14T07:07:54.900859+0000","last_clean":"2024-09-14T07:07:54.900859+0000","last_became_active":"2024-09-14T07:07:36.196923+0000","last_became_peered":"2024-09-14T07:07:36.196923+0000","last_unstale":"2024-09-14T07:07:54.900859+0000","last_undegraded":"2024-09-14T07:07:54.900859+0000","last_fullsized":"2024-09-14T07:07:54.900859+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:20.173745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"26'6","reported_seq":463,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:55.145319+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:07:55.145319+0000","last_peered":"2024-09-14T07:07:55.145319+0000","last_clean":"2024-09-14T07:07:55.145319+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:07:55.145319+0000","last_undegraded":"2024-09-14T07:07:55.145319+0000","last_fullsized":"2024-09-14T07:07:55.145319+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:15:34.023780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":4629504,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":329,"num_read_kb":2208,"num_write":6,"num_write_kb":205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"24'2","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T07:07:55.840544+0000","last_change":"2024-09-14T07:07:49.808620+0000","last_active":"2024-09-14T07:07:55.840544+0000","last_peered":"2024-09-14T07:07:55.840544+0000","last_clean":"2024-09-14T07:07:55.840544+0000","last_became_active":"2024-09-14T07:07:47.786098+0000","last_became_peered":"2024-09-14T07:07:47.786098+0000","last_unstale":"2024-09-14T07:07:55.840544+0000","last_undegraded":"2024-09-14T07:07:55.840544+0000","last_fullsized":"2024-09-14T07:07:55.840544+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:14.279103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.831446+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:07:54.831446+0000","last_peered":"2024-09-14T07:07:54.831446+0000","last_clean":"2024-09-14T07:07:54.831446+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:07:54.831446+0000","last_undegraded":"2024-09-14T07:07:54.831446+0000","last_fullsized":"2024-09-14T07:07:54.831446+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.900547+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:07:54.900547+0000","last_peered":"2024-09-14T07:07:54.900547+0000","last_clean":"2024-09-14T07:07:54.900547+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:07:54.900547+0000","last_undegraded":"2024-09-14T07:07:54.900547+0000","last_fullsized":"2024-09-14T07:07:54.900547+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:18:38.539525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.845875+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:07:54.845875+0000","last_peered":"2024-09-14T07:07:54.845875+0000","last_clean":"2024-09-14T07:07:54.845875+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:07:54.845875+0000","last_undegraded":"2024-09-14T07:07:54.845875+0000","last_fullsized":"2024-09-14T07:07:54.845875+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"24'4","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.827234+0000","last_change":"2024-09-14T07:07:49.801096+0000","last_active":"2024-09-14T07:07:54.827234+0000","last_peered":"2024-09-14T07:07:54.827234+0000","last_clean":"2024-09-14T07:07:54.827234+0000","last_became_active":"2024-09-14T07:07:47.789827+0000","last_became_peered":"2024-09-14T07:07:47.789827+0000","last_unstale":"2024-09-14T07:07:54.827234+0000","last_undegraded":"2024-09-14T07:07:54.827234+0000","last_fullsized":"2024-09-14T07:07:54.827234+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:30:43.198672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":244,"num_write":4,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.826252+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:07:54.826252+0000","last_peered":"2024-09-14T07:07:54.826252+0000","last_clean":"2024-09-14T07:07:54.826252+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:07:54.826252+0000","last_undegraded":"2024-09-14T07:07:54.826252+0000","last_fullsized":"2024-09-14T07:07:54.826252+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'13","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:55.193463+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:07:55.193463+0000","last_peered":"2024-09-14T07:07:55.193463+0000","last_clean":"2024-09-14T07:07:55.193463+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:07:55.193463+0000","last_undegraded":"2024-09-14T07:07:55.193463+0000","last_fullsized":"2024-09-14T07:07:55.193463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:45.391084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":93,"num_write":13,"num_write_kb":203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.832758+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:07:54.832758+0000","last_peered":"2024-09-14T07:07:54.832758+0000","last_clean":"2024-09-14T07:07:54.832758+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:07:54.832758+0000","last_undegraded":"2024-09-14T07:07:54.832758+0000","last_fullsized":"2024-09-14T07:07:54.832758+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":127,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.826358+0000","last_change":"2024-09-14T07:07:32.604869+0000","last_active":"2024-09-14T07:07:54.826358+0000","last_peered":"2024-09-14T07:07:54.826358+0000","last_clean":"2024-09-14T07:07:54.826358+0000","last_became_active":"2024-09-14T07:07:32.602241+0000","last_became_peered":"2024-09-14T07:07:32.602241+0000","last_unstale":"2024-09-14T07:07:54.826358+0000","last_undegraded":"2024-09-14T07:07:54.826358+0000","last_fullsized":"2024-09-14T07:07:54.826358+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:25.905190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"26'4","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.845734+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:07:54.845734+0000","last_peered":"2024-09-14T07:07:54.845734+0000","last_clean":"2024-09-14T07:07:54.845734+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:07:54.845734+0000","last_undegraded":"2024-09-14T07:07:54.845734+0000","last_fullsized":"2024-09-14T07:07:54.845734+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:20:13.534465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":4106752,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"27'140","reported_seq":181,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-14T07:07:55.840323+0000","last_change":"2024-09-14T07:07:53.858015+0000","last_active":"2024-09-14T07:07:55.840323+0000","last_peered":"2024-09-14T07:07:55.840323+0000","last_clean":"2024-09-14T07:07:55.840323+0000","last_became_active":"2024-09-14T07:07:53.839838+0000","last_became_peered":"2024-09-14T07:07:53.839838+0000","last_unstale":"2024-09-14T07:07:55.840323+0000","last_undegraded":"2024-09-14T07:07:55.840323+0000","last_fullsized":"2024-09-14T07:07:55.840323+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:34:42.885991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":49,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":687,"num_read_kb":549,"num_write":232,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":47,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"27'14","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.827040+0000","last_change":"2024-09-14T07:07:49.803037+0000","last_active":"2024-09-14T07:07:54.827040+0000","last_peered":"2024-09-14T07:07:54.827040+0000","last_clean":"2024-09-14T07:07:54.827040+0000","last_became_active":"2024-09-14T07:07:47.801723+0000","last_became_peered":"2024-09-14T07:07:47.801723+0000","last_unstale":"2024-09-14T07:07:54.827040+0000","last_undegraded":"2024-09-14T07:07:54.827040+0000","last_fullsized":"2024-09-14T07:07:54.827040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:08.820620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":1452032,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":9,"num_write":14,"num_write_kb":294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.900694+0000","last_change":"2024-09-14T07:07:49.813689+0000","last_active":"2024-09-14T07:07:54.900694+0000","last_peered":"2024-09-14T07:07:54.900694+0000","last_clean":"2024-09-14T07:07:54.900694+0000","last_became_active":"2024-09-14T07:07:47.803551+0000","last_became_peered":"2024-09-14T07:07:47.803551+0000","last_unstale":"2024-09-14T07:07:54.900694+0000","last_undegraded":"2024-09-14T07:07:54.900694+0000","last_fullsized":"2024-09-14T07:07:54.900694+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:33:44.649755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"24'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.831925+0000","last_change":"2024-09-14T07:07:49.820702+0000","last_active":"2024-09-14T07:07:54.831925+0000","last_peered":"2024-09-14T07:07:54.831925+0000","last_clean":"2024-09-14T07:07:54.831925+0000","last_became_active":"2024-09-14T07:07:47.781704+0000","last_became_peered":"2024-09-14T07:07:47.781704+0000","last_unstale":"2024-09-14T07:07:54.831925+0000","last_undegraded":"2024-09-14T07:07:54.831925+0000","last_fullsized":"2024-09-14T07:07:54.831925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:46:06.196387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"26'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.827050+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:07:54.827050+0000","last_peered":"2024-09-14T07:07:54.827050+0000","last_clean":"2024-09-14T07:07:54.827050+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:07:54.827050+0000","last_undegraded":"2024-09-14T07:07:54.827050+0000","last_fullsized":"2024-09-14T07:07:54.827050+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:02:31.819235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":8,"num_write":3,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"24'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.832023+0000","last_change":"2024-09-14T07:07:49.820959+0000","last_active":"2024-09-14T07:07:54.832023+0000","last_peered":"2024-09-14T07:07:54.832023+0000","last_clean":"2024-09-14T07:07:54.832023+0000","last_became_active":"2024-09-14T07:07:47.786208+0000","last_became_peered":"2024-09-14T07:07:47.786208+0000","last_unstale":"2024-09-14T07:07:54.832023+0000","last_undegraded":"2024-09-14T07:07:54.832023+0000","last_fullsized":"2024-09-14T07:07:54.832023+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:50.869115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.832594+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:07:54.832594+0000","last_peered":"2024-09-14T07:07:54.832594+0000","last_clean":"2024-09-14T07:07:54.832594+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:07:54.832594+0000","last_undegraded":"2024-09-14T07:07:54.832594+0000","last_fullsized":"2024-09-14T07:07:54.832594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:59:49.964093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"24'4","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.831583+0000","last_change":"2024-09-14T07:07:49.818154+0000","last_active":"2024-09-14T07:07:54.831583+0000","last_peered":"2024-09-14T07:07:54.831583+0000","last_clean":"2024-09-14T07:07:54.831583+0000","last_became_active":"2024-09-14T07:07:47.782423+0000","last_became_peered":"2024-09-14T07:07:47.782423+0000","last_unstale":"2024-09-14T07:07:54.831583+0000","last_undegraded":"2024-09-14T07:07:54.831583+0000","last_fullsized":"2024-09-14T07:07:54.831583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:03.185980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"26'6","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.826401+0000","last_change":"2024-09-14T07:07:49.802393+0000","last_active":"2024-09-14T07:07:54.826401+0000","last_peered":"2024-09-14T07:07:54.826401+0000","last_clean":"2024-09-14T07:07:54.826401+0000","last_became_active":"2024-09-14T07:07:47.791132+0000","last_became_peered":"2024-09-14T07:07:47.791132+0000","last_unstale":"2024-09-14T07:07:54.826401+0000","last_undegraded":"2024-09-14T07:07:54.826401+0000","last_fullsized":"2024-09-14T07:07:54.826401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:37.921590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":3663872,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":50,"num_write":6,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"26'11","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.827373+0000","last_change":"2024-09-14T07:07:49.800610+0000","last_active":"2024-09-14T07:07:54.827373+0000","last_peered":"2024-09-14T07:07:54.827373+0000","last_clean":"2024-09-14T07:07:54.827373+0000","last_became_active":"2024-09-14T07:07:47.795686+0000","last_became_peered":"2024-09-14T07:07:47.795686+0000","last_unstale":"2024-09-14T07:07:54.827373+0000","last_undegraded":"2024-09-14T07:07:54.827373+0000","last_fullsized":"2024-09-14T07:07:54.827373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:13:06.194598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":10882048,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":126,"num_write":11,"num_write_kb":205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.13","version":"26'10","reported_seq":85,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T07:07:54.826693+0000","last_change":"2024-09-14T07:07:49.804715+0000","last_active":"2024-09-14T07:07:54.826693+0000","last_peered":"2024-09-14T07:07:54.826693+0000","last_clean":"2024-09-14T07:07:54.826693+0000","last_became_active":"2024-09-14T07:07:47.787058+0000","last_became_peered":"2024-09-14T07:07:47.787058+0000","last_unstale":"2024-09-14T07:07:54.826693+0000","last_undegraded":"2024-09-14T07:07:54.826693+0000","last_fullsized":"2024-09-14T07:07:54.826693+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:15:32.474124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":385,"num_write":10,"num_write_kb":176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":35240635,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":5720,"num_read_kb":8290,"num_write":451,"num_write_kb":2951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":139311,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":1937270,"data_compressed":444198,"data_compressed_allocated":593920,"data_compressed_original":1867776,"omap_allocated":0,"internal_metadata":0},"log_size":447,"ondisk_log_size":447,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35696,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336144,"statfs":{"total":96636764160,"available":96600211456,"internally_reserved":0,"allocated":360448,"data_stored":340800,"data_compressed":27534,"data_compressed_allocated":45056,"data_compressed_original":229376,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607561,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35684,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336156,"statfs":{"total":96636764160,"available":96600223744,"internally_reserved":0,"allocated":335872,"data_stored":283909,"data_compressed":32306,"data_compressed_allocated":45056,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607560,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":487424,"data_stored":1061870,"data_compressed":91289,"data_compressed_allocated":155648,"data_compressed_original":933888,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607562,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":409600,"data_stored":502051,"data_compressed":68930,"data_compressed_allocated":98304,"data_compressed_original":389120,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607562,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":376832,"data_stored":310761,"data_compressed":46844,"data_compressed_allocated":61440,"data_compressed_original":200704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607562,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":454656,"data_stored":898030,"data_compressed":82194,"data_compressed_allocated":131072,"data_compressed_original":774144,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":206913,"data_compressed":76136,"data_compressed_allocated":94208,"data_compressed_original":200704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":209993,"data_compressed":46844,"data_compressed_allocated":61440,"data_compressed_original":200704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":406577,"data_compressed":68930,"data_compressed_allocated":98304,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":151552,"data_stored":370753,"data_compressed":85231,"data_compressed_allocated":118784,"data_compressed_original":360448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":554602,"data_compressed":134751,"data_compressed_allocated":176128,"data_compressed_original":536576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":188432,"data_compressed":32306,"data_compressed_allocated":45056,"data_compressed_original":180224,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:07:58.630 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [4, 2, 5, 1] 2024-09-14T07:07:58.631 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.5', '4', '2', '5', '1'] 2024-09-14T07:07:58.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.5 4 2 5 1 2024-09-14T07:07:58.711 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:07:58.711 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:58.711 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:58.710+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:58.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:07:58.823 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:58.822+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:07:58.881 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:58.878+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:58.917 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:07:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:07:58.917 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:58.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:07:58.982 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:58.978+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:59.083 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:59.078+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:59.103 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:07:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:07:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:07:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:20 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'10, mlcod=14'8)", 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.455572523000001, 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002918571, 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645365+0000", 2024-09-14T07:07:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7799999999999999e-06 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645608+0000", 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024235200000000001 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645637+0000", 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9315999999999998e-05 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645651+0000", 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.375e-05 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645777+0000", 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012618199999999999 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645830+0000", 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3340000000000001e-05 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648125+0000", 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022950549999999998 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648282+0000", 2024-09-14T07:07:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015679600000000001 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:21 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'11, mlcod=14'8)", 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.452985199, 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041484010000000003, 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647953+0000", 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.79e-06 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648081+0000", 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000128102 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648114+0000", 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3269000000000003e-05 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648150+0000", 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5939999999999998e-05 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648364+0000", 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000214415 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648418+0000", 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3386e-05 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652012+0000", 2024-09-14T07:07:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035942700000000001 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652099+0000", 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7229000000000001e-05 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:22 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'12, mlcod=14'8)", 2024-09-14T07:07:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.450724872999999, 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041167649999999997, 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650213+0000", 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7129999999999999e-06 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650325+0000", 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011184900000000001 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650354+0000", 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9595999999999999e-05 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650385+0000", 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0179999999999999e-05 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650512+0000", 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012745899999999999 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650567+0000", 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4447000000000002e-05 2024-09-14T07:07:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654240+0000", 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036733109999999999 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654328+0000", 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8209999999999997e-05 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:65 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'43, mlcod=14'40)", 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.142105561000001, 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032602360000000001, 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958832+0000", 2024-09-14T07:07:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0249999999999999e-06 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958839+0000", 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6979999999999998e-06 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958857+0000", 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958865+0000", 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958964+0000", 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8787000000000001e-05 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959008+0000", 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3720000000000002e-05 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962018+0000", 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030102599999999998 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962091+0000", 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2985999999999995e-05 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:66 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'44, mlcod=14'40)", 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.141517914000001, 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045205219999999999, 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959419+0000", 2024-09-14T07:07:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959471+0000", 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1078000000000001e-05 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959484+0000", 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3083e-05 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959493+0000", 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5540000000000003e-06 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959682+0000", 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018848700000000001 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959722+0000", 2024-09-14T07:07:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0784000000000001e-05 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963902+0000", 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041791110000000001 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963939+0000", 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7212000000000002e-05 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:67 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'45, mlcod=14'40)", 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.14036437, 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044867969999999998, 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960573+0000", 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8100000000000001e-07 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960582+0000", 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960598+0000", 2024-09-14T07:07:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6184999999999998e-05 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960605+0000", 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3320000000000003e-06 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961328+0000", 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000723075 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961373+0000", 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4552000000000003e-05 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965024+0000", 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036513520000000001 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965059+0000", 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4430000000000001e-05 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'46, mlcod=14'41)", 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.139620059999999, 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003788769, 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961317+0000", 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.02e-06 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961326+0000", 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5539999999999998e-06 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961341+0000", 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5e-05 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961348+0000", 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8310000000000001e-06 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962174+0000", 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082531300000000004 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962220+0000", 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6338000000000003e-05 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965063+0000", 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028425590000000001 2024-09-14T07:07:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965105+0000", 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2154000000000003e-05 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:69 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'47, mlcod=14'41)", 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:07:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.138999185999999, 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032062459999999998, 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961938+0000", 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961974+0000", 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6205000000000003e-05 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961993+0000", 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8286000000000001e-05 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962000+0000", 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4279999999999999e-06 2024-09-14T07:07:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962847+0000", 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084736100000000003 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962865+0000", 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8028000000000001e-05 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965112+0000", 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022461199999999999 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965143+0000", 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1687999999999999e-05 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:71 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'49, mlcod=14'42)", 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.137313367000001, 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036766120000000001, 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963624+0000", 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3900000000000003e-07 2024-09-14T07:07:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963632+0000", 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8359999999999992e-06 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963650+0000", 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8130000000000001e-05 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963659+0000", 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2779999999999999e-06 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964704+0000", 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001044683 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964720+0000", 2024-09-14T07:07:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6456e-05 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967251+0000", 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002531307 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967299+0000", 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7982999999999998e-05 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:72 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'50, mlcod=14'42)", 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.136724264000001, 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046904399999999997, 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964213+0000", 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8400000000000003e-07 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964294+0000", 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0747000000000005e-05 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964306+0000", 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2155e-05 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964314+0000", 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0090000000000007e-06 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965268+0000", 2024-09-14T07:07:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095471900000000003 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965316+0000", 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7117000000000003e-05 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968866+0000", 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003550311 2024-09-14T07:07:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968902+0000", 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6498000000000002e-05 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:73 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'51, mlcod=14'42)", 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.135511528999999, 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035339820000000002, 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965441+0000", 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6181999999999999e-05 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965453+0000", 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1252e-05 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965460+0000", 2024-09-14T07:07:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966961+0000", 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015005929999999999 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966978+0000", 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6991999999999999e-05 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968907+0000", 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019290329999999999 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968959+0000", 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2033999999999997e-05 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:78 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'56, mlcod=14'53)", 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.128988536000001, 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046602329999999997, 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:07:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972013+0000", 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.444e-05 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972026+0000", 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3497e-05 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972033+0000", 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.533e-06 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972762+0000", 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072933500000000001 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972782+0000", 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9417000000000001e-05 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976515+0000", 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037336370000000002 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976608+0000", 2024-09-14T07:07:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2645999999999994e-05 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:79 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'57, mlcod=14'53)", 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.127981174999999, 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003768225, 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972956+0000", 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3599999999999999e-06 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973052+0000", 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6132999999999996e-05 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973072+0000", 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9307999999999999e-05 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973089+0000", 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7569000000000001e-05 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973597+0000", 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00050773500000000004 2024-09-14T07:07:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973646+0000", 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8773000000000002e-05 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976625+0000", 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002979607 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976723+0000", 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7739999999999996e-05 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:80 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'58, mlcod=14'53)", 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.127055980000002, 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030979940000000002, 2024-09-14T07:07:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:07:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973881+0000", 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973963+0000", 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2442999999999997e-05 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973975+0000", 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1595e-05 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973982+0000", 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7789999999999998e-06 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.975181+0000", 2024-09-14T07:07:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001199809 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.975228+0000", 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6097999999999998e-05 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976917+0000", 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001689091 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976978+0000", 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1501999999999995e-05 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1005 3.15 e26/21 3:aa92e940:::rbd_data.11611b5ca054.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1865982050000001, 2024-09-14T07:07:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094907240000000007, 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4700000000000002e-07 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914538+0000", 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020003100000000001 2024-09-14T07:07:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914567+0000", 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8153e-05 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914582+0000", 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5194000000000001e-05 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914727+0000", 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000145205 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914769+0000", 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1797e-05 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.923737+0000", 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.008968228 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.923829+0000", 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1669000000000006e-05 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1153 3.18 e26/25 3:1f883cd2:::rbd_data.11611b5ca054.000000000000001c:head v 26'3, mlcod=0'0)", 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.0753308639999997, 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012967347000000001, 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025607+0000", 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0169999999999999e-06 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025708+0000", 2024-09-14T07:07:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010069900000000001 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025739+0000", 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1086e-05 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025759+0000", 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-05 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025880+0000", 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120623 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025918+0000", 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8206e-05 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038441+0000", 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012523117 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038573+0000", 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013165900000000001 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1268 3.6 3:6107e7ff:::rbd_data.11611b5ca054.0000000000000034:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4071424~54272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.116008+0000", 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9849283350000002, 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045734210000000003, 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1268 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116008+0000", 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116008+0000", 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116011+0000", 2024-09-14T07:07:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0299999999999998e-06 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116207+0000", 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019582200000000001 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116240+0000", 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3312999999999998e-05 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116255+0000", 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5247e-05 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116421+0000", 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000165468 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116718+0000", 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00029703100000000001 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117046+0000", 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000328661 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.119249+0000", 2024-09-14T07:07:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002202613 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.120406+0000", 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001157307 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.120503+0000", 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7173e-05 2024-09-14T07:07:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.120581+0000", 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7756000000000006e-05 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1356 3.12 3:49b07ccb:::rbd_data.11611b5ca054.0000000000000009:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3604480~24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.184176+0000", 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9167599840000005, 2024-09-14T07:07:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005573433, 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1356 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184176+0000", 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184176+0000", 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184179+0000", 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.39e-06 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184213+0000", 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4125999999999998e-05 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184246+0000", 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3287000000000002e-05 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184268+0000", 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2123000000000001e-05 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184437+0000", 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016902200000000001 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184816+0000", 2024-09-14T07:07:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00037903299999999997 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.185264+0000", 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00044744400000000002 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189042+0000", 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037787839999999999 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189564+0000", 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00052154199999999995 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189669+0000", 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010473 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189750+0000", 2024-09-14T07:07:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0951999999999993e-05 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1426 3.12 3:49b07ccb:::rbd_data.11611b5ca054.0000000000000009:head [write 2047488~35328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.253214+0000", 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8477226299999998, 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029816259999999998, 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1426 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253214+0000", 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253214+0000", 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253216+0000", 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253255+0000", 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8940000000000003e-05 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253287+0000", 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1965999999999999e-05 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253304+0000", 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6747000000000001e-05 2024-09-14T07:07:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253433+0000", 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000129629 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253565+0000", 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013143200000000001 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253772+0000", 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020675199999999999 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.255736+0000", 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019647250000000001 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.255869+0000", 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000132705 2024-09-14T07:07:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.256046+0000", 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176956 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.256195+0000", 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014919799999999999 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1536 3.15 e26/21 3:aa92e940:::rbd_data.11611b5ca054.0000000000000013:head v 26'4, mlcod=24'2)", 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.320416+0000", 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7805204290000001, 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0039702920000000003, 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320416+0000", 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320416+0000", 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320418+0000", 2024-09-14T07:07:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8470000000000001e-06 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320451+0000", 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3939999999999997e-05 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320481+0000", 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9149000000000002e-05 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320496+0000", 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5319999999999999e-05 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320651+0000", 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015520900000000001 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320691+0000", 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0009999999999998e-05 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.324299+0000", 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036082750000000002 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.324386+0000", 2024-09-14T07:07:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6541999999999999e-05 2024-09-14T07:07:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:07:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:07:59.213 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:59.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:07:59.215 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:59.214+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:59.315 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:07:59.314+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:07:59.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:07:59.388 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:07:59.388 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:07:59.389 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:07:59.389 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:07:59.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:07:59.416 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:59.414+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:59.517 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:07:59.514+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:07:59.519 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:07:59.536 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:07:59.537 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:07:59.553 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:07:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:07:59.561 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:59.561 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:59.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:07:59.576 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:07:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:07:59.594 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:07:59.595 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:07:59.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:07:59.616 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:07:59.618 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:59.614+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:59.719 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:07:59.718+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:07:59.722 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:59.722 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:59.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:07:59.790 INFO:teuthology.orchestra.run.smithi145.stdout:459 trunc from 0xcd18a00 to 0xe5a1e00 2024-09-14T07:07:59.820 INFO:teuthology.orchestra.run.smithi145.stdout:462 punch from 0xa9ee800 to 0xa9eee00, (0x600 bytes) 2024-09-14T07:07:59.821 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:07:59.818+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:07:59.823 INFO:teuthology.orchestra.run.smithi145.stdout:463 punch from 0xc354600 to 0xc35aa00, (0x6400 bytes) 2024-09-14T07:07:59.824 INFO:teuthology.orchestra.run.smithi145.stdout:468 punch from 0x4445000 to 0x4450a00, (0xba00 bytes) 2024-09-14T07:07:59.824 INFO:teuthology.orchestra.run.smithi145.stdout:472 read 0x3ac200 thru 0x3bbdb3 (0xfbb4 bytes) 2024-09-14T07:07:59.867 INFO:teuthology.orchestra.run.smithi145.stdout:473 trunc from 0xe5a1e00 to 0x6017c00 2024-09-14T07:07:59.867 INFO:teuthology.orchestra.run.smithi005.stderr:set 2.5 pg_upmap_items mapping to [4->2,5->1] 2024-09-14T07:07:59.877 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:07:59.877 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:07:59.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:07:59.922 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:07:59.918+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:07:59.924 INFO:teuthology.orchestra.run.smithi145.stdout:474 read 0x3069400 thru 0x306c54c (0x314d bytes) 2024-09-14T07:07:59.928 INFO:teuthology.orchestra.run.smithi145.stdout:476 read 0x296c400 thru 0x296de94 (0x1a95 bytes) 2024-09-14T07:07:59.929 INFO:teuthology.orchestra.run.smithi145.stdout:477 write 0x36c7e00 thru 0x36ce15f (0x6360 bytes) 2024-09-14T07:07:59.976 INFO:teuthology.orchestra.run.smithi145.stdout:478 write 0xda72c00 thru 0xda78694 (0x5a95 bytes) 2024-09-14T07:08:00.022 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:00.018+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:00.048 INFO:teuthology.orchestra.run.smithi145.stdout:479 trunc from 0xda78800 to 0x118e000 2024-09-14T07:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:00.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:00.126 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:00.126+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:00.150 INFO:teuthology.orchestra.run.smithi145.stdout:480 write 0x71a5600 thru 0x71af457 (0x9e58 bytes) 2024-09-14T07:08:00.151 INFO:teuthology.orchestra.run.smithi145.stdout:482 punch from 0x15a3000 to 0x15b2c00, (0xfc00 bytes) 2024-09-14T07:08:00.152 INFO:teuthology.orchestra.run.smithi145.stdout:483 read 0x2ed8400 thru 0x2ee66dd (0xe2de bytes) 2024-09-14T07:08:00.152 INFO:teuthology.orchestra.run.smithi145.stdout:485 clone 3 order 19 su 65536 sc 10 2024-09-14T07:08:00.206 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:00.206 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:00.221 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:00.224 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:00.222+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:00.325 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:00.322+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:00.353 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:00.353 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:00.367 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:00.425 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:00.422+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:00.490 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:00.491 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:00.505 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:00.551 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:00.550+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:00.623 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:00.623 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:00.627 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:00.622+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:00.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:00.728 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:00.726+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:00.809 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:00.823 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:00.828 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:00.826+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:00.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:08:00.931 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:00.930+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:01.010 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:01.010 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:01.017 INFO:teuthology.orchestra.run.smithi145.stdout:leaving image image_client.0-clone2 intact 2024-09-14T07:08:01.025 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:01.029 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:01.026+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:01.131 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:01.130+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:01.148 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:01.148 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:01.162 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:01.231 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:01.230+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:01.289 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:01.289 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:01.303 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:01.351 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:01.350+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:01.433 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:01.430+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:01.479 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:01.479 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:01.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:01.534 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:01.530+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:01.548 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T07:08:01.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:01.635 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:01.634+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:01.654 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:01.654 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:01.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:01.736 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:01.734+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:01.805 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:01.818 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:01.818 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:01.818 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:01.819 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:01.824 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:01.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:01.843 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:01.844 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:01.844 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:01.842+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:01.866 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:01.867 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:01.877 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:08:01.891 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:01.915 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:01.938 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:01.934+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:01.970 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:01.971 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:01.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:02.074 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:02.074+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:02.140 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:02.138+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:02.164 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:20 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'10, mlcod=14'8)", 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.517473085999999, 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002918571, 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645364+0000", 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645365+0000", 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7799999999999999e-06 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645608+0000", 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024235200000000001 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645637+0000", 2024-09-14T07:08:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9315999999999998e-05 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645651+0000", 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.375e-05 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645777+0000", 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012618199999999999 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.645830+0000", 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3340000000000001e-05 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648125+0000", 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022950549999999998 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648282+0000", 2024-09-14T07:08:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015679600000000001 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:21 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'11, mlcod=14'8)", 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.514885761999999, 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041484010000000003, 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647951+0000", 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.647953+0000", 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.79e-06 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648081+0000", 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000128102 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648114+0000", 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3269000000000003e-05 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648150+0000", 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5939999999999998e-05 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648364+0000", 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000214415 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.648418+0000", 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3386e-05 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652012+0000", 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035942700000000001 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.652099+0000", 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7229000000000001e-05 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:22 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'12, mlcod=14'8)", 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.512625436, 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041167649999999997, 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650211+0000", 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650213+0000", 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7129999999999999e-06 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650325+0000", 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011184900000000001 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650354+0000", 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9595999999999999e-05 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650385+0000", 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0179999999999999e-05 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650512+0000", 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012745899999999999 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.650567+0000", 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4447000000000002e-05 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654240+0000", 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036733109999999999 2024-09-14T07:08:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.654328+0000", 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8209999999999997e-05 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:65 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'43, mlcod=14'40)", 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.204006123999999, 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032602360000000001, 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958831+0000", 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958832+0000", 2024-09-14T07:08:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0249999999999999e-06 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958839+0000", 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6979999999999998e-06 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958857+0000", 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958865+0000", 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.958964+0000", 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8787000000000001e-05 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959008+0000", 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3720000000000002e-05 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962018+0000", 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030102599999999998 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962091+0000", 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2985999999999995e-05 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:66 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'44, mlcod=14'40)", 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.203418477, 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045205219999999999, 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959418+0000", 2024-09-14T07:08:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959419+0000", 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959471+0000", 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1078000000000001e-05 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959484+0000", 2024-09-14T07:08:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3083e-05 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959493+0000", 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5540000000000003e-06 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959682+0000", 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018848700000000001 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.959722+0000", 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0784000000000001e-05 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963902+0000", 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041791110000000001 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963939+0000", 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7212000000000002e-05 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:67 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'45, mlcod=14'40)", 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.202264932999999, 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044867969999999998, 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:08:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960572+0000", 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960573+0000", 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8100000000000001e-07 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960582+0000", 2024-09-14T07:08:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960598+0000", 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6184999999999998e-05 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.960605+0000", 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3320000000000003e-06 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961328+0000", 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000723075 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961373+0000", 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4552000000000003e-05 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965024+0000", 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036513520000000001 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965059+0000", 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4430000000000001e-05 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'46, mlcod=14'41)", 2024-09-14T07:08:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:08:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.201520623, 2024-09-14T07:08:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003788769, 2024-09-14T07:08:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:08:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961316+0000", 2024-09-14T07:08:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961317+0000", 2024-09-14T07:08:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.02e-06 2024-09-14T07:08:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961326+0000", 2024-09-14T07:08:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5539999999999998e-06 2024-09-14T07:08:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961341+0000", 2024-09-14T07:08:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5e-05 2024-09-14T07:08:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961348+0000", 2024-09-14T07:08:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8310000000000001e-06 2024-09-14T07:08:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962174+0000", 2024-09-14T07:08:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082531300000000004 2024-09-14T07:08:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962220+0000", 2024-09-14T07:08:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6338000000000003e-05 2024-09-14T07:08:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965063+0000", 2024-09-14T07:08:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028425590000000001 2024-09-14T07:08:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965105+0000", 2024-09-14T07:08:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2154000000000003e-05 2024-09-14T07:08:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:69 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'47, mlcod=14'41)", 2024-09-14T07:08:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:08:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.200899749000001, 2024-09-14T07:08:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032062459999999998, 2024-09-14T07:08:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:08:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961937+0000", 2024-09-14T07:08:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961938+0000", 2024-09-14T07:08:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-09-14T07:08:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961974+0000", 2024-09-14T07:08:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6205000000000003e-05 2024-09-14T07:08:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.961993+0000", 2024-09-14T07:08:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8286000000000001e-05 2024-09-14T07:08:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962000+0000", 2024-09-14T07:08:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4279999999999999e-06 2024-09-14T07:08:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962847+0000", 2024-09-14T07:08:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084736100000000003 2024-09-14T07:08:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.962865+0000", 2024-09-14T07:08:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8028000000000001e-05 2024-09-14T07:08:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965112+0000", 2024-09-14T07:08:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022461199999999999 2024-09-14T07:08:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965143+0000", 2024-09-14T07:08:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1687999999999999e-05 2024-09-14T07:08:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:71 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'49, mlcod=14'42)", 2024-09-14T07:08:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:08:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.199213929999999, 2024-09-14T07:08:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036766120000000001, 2024-09-14T07:08:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:08:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963623+0000", 2024-09-14T07:08:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963624+0000", 2024-09-14T07:08:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3900000000000003e-07 2024-09-14T07:08:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963632+0000", 2024-09-14T07:08:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8359999999999992e-06 2024-09-14T07:08:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963650+0000", 2024-09-14T07:08:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8130000000000001e-05 2024-09-14T07:08:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.963659+0000", 2024-09-14T07:08:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2779999999999999e-06 2024-09-14T07:08:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964704+0000", 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001044683 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964720+0000", 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6456e-05 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967251+0000", 2024-09-14T07:08:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002531307 2024-09-14T07:08:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.967299+0000", 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7982999999999998e-05 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:72 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'50, mlcod=14'42)", 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:08:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.198624827, 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046904399999999997, 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964212+0000", 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964213+0000", 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8400000000000003e-07 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964294+0000", 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0747000000000005e-05 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964306+0000", 2024-09-14T07:08:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2155e-05 2024-09-14T07:08:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.964314+0000", 2024-09-14T07:08:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0090000000000007e-06 2024-09-14T07:08:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965268+0000", 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095471900000000003 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965316+0000", 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7117000000000003e-05 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968866+0000", 2024-09-14T07:08:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003550311 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968902+0000", 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6498000000000002e-05 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:73 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'51, mlcod=14'42)", 2024-09-14T07:08:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.197412092, 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035339820000000002, 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965425+0000", 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965441+0000", 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6181999999999999e-05 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965453+0000", 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1252e-05 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.965460+0000", 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966961+0000", 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015005929999999999 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.966978+0000", 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6991999999999999e-05 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968907+0000", 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019290329999999999 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.968959+0000", 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2033999999999997e-05 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.232 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:78 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'56, mlcod=14'53)", 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.190889099, 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046602329999999997, 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.971948+0000", 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972013+0000", 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.444e-05 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972026+0000", 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3497e-05 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972033+0000", 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.533e-06 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972762+0000", 2024-09-14T07:08:02.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072933500000000001 2024-09-14T07:08:02.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972782+0000", 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9417000000000001e-05 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976515+0000", 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037336370000000002 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976608+0000", 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2645999999999994e-05 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.237 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:79 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'57, mlcod=14'53)", 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.189881738, 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003768225, 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:08:02.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972955+0000", 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.972956+0000", 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3599999999999999e-06 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973052+0000", 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6132999999999996e-05 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973072+0000", 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9307999999999999e-05 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973089+0000", 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7569000000000001e-05 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973597+0000", 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00050773500000000004 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973646+0000", 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8773000000000002e-05 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976625+0000", 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002979607 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976723+0000", 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7739999999999996e-05 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4177.0:80 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'58, mlcod=14'53)", 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.188956543, 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030979940000000002, 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973880+0000", 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973881+0000", 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973963+0000", 2024-09-14T07:08:02.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2442999999999997e-05 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973975+0000", 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1595e-05 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.973982+0000", 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7789999999999998e-06 2024-09-14T07:08:02.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.975181+0000", 2024-09-14T07:08:02.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001199809 2024-09-14T07:08:02.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.975228+0000", 2024-09-14T07:08:02.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6097999999999998e-05 2024-09-14T07:08:02.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976917+0000", 2024-09-14T07:08:02.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001689091 2024-09-14T07:08:02.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:32.976978+0000", 2024-09-14T07:08:02.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1501999999999995e-05 2024-09-14T07:08:02.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1005 3.15 e26/21 3:aa92e940:::rbd_data.11611b5ca054.0000000000000013:head v 26'3, mlcod=24'1)", 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2484987679999993, 2024-09-14T07:08:02.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094907240000000007, 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:08:02.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:08:02.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914338+0000", 2024-09-14T07:08:02.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4700000000000002e-07 2024-09-14T07:08:02.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914538+0000", 2024-09-14T07:08:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020003100000000001 2024-09-14T07:08:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914567+0000", 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8153e-05 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914582+0000", 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5194000000000001e-05 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914727+0000", 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000145205 2024-09-14T07:08:02.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.914769+0000", 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1797e-05 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.923737+0000", 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.008968228 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.923829+0000", 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1669000000000006e-05 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1153 3.18 e26/25 3:1f883cd2:::rbd_data.11611b5ca054.000000000000001c:head v 26'3, mlcod=0'0)", 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1372314269999997, 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012967347000000001, 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:08:02.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025607+0000", 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0169999999999999e-06 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025708+0000", 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010069900000000001 2024-09-14T07:08:02.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025739+0000", 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1086e-05 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025759+0000", 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-05 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025880+0000", 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120623 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025918+0000", 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8206e-05 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038441+0000", 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012523117 2024-09-14T07:08:02.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038573+0000", 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013165900000000001 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1268 3.6 3:6107e7ff:::rbd_data.11611b5ca054.0000000000000034:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4071424~54272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.116008+0000", 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0468288979999993, 2024-09-14T07:08:02.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045734210000000003, 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1268 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116008+0000", 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116008+0000", 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116011+0000", 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0299999999999998e-06 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116207+0000", 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019582200000000001 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116240+0000", 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3312999999999998e-05 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116255+0000", 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5247e-05 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116421+0000", 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000165468 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.116718+0000", 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00029703100000000001 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117046+0000", 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000328661 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.119249+0000", 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002202613 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.120406+0000", 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001157307 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.120503+0000", 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7173e-05 2024-09-14T07:08:02.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.120581+0000", 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7756000000000006e-05 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1356 3.12 3:49b07ccb:::rbd_data.11611b5ca054.0000000000000009:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3604480~24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.184176+0000", 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.9786605470000005, 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005573433, 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1356 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184176+0000", 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184176+0000", 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184179+0000", 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.39e-06 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184213+0000", 2024-09-14T07:08:02.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4125999999999998e-05 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184246+0000", 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3287000000000002e-05 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184268+0000", 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2123000000000001e-05 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184437+0000", 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016902200000000001 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.184816+0000", 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00037903299999999997 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.185264+0000", 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00044744400000000002 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189042+0000", 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037787839999999999 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189564+0000", 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00052154199999999995 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189669+0000", 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010473 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.189750+0000", 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0951999999999993e-05 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1426 3.12 3:49b07ccb:::rbd_data.11611b5ca054.0000000000000009:head [write 2047488~35328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.253214+0000", 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.9096231929999998, 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029816259999999998, 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1426 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253214+0000", 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253214+0000", 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253216+0000", 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253255+0000", 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8940000000000003e-05 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253287+0000", 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1965999999999999e-05 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253304+0000", 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6747000000000001e-05 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253433+0000", 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000129629 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253565+0000", 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013143200000000001 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253772+0000", 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020675199999999999 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.255736+0000", 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019647250000000001 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.255869+0000", 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000132705 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.256046+0000", 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176956 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.256195+0000", 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014919799999999999 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1536 3.15 e26/21 3:aa92e940:::rbd_data.11611b5ca054.0000000000000013:head v 26'4, mlcod=24'2)", 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.320416+0000", 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.8424209920000001, 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0039702920000000003, 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320416+0000", 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320416+0000", 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320418+0000", 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8470000000000001e-06 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320451+0000", 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3939999999999997e-05 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320481+0000", 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9149000000000002e-05 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320496+0000", 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5319999999999999e-05 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320651+0000", 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015520900000000001 2024-09-14T07:08:02.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.320691+0000", 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0009999999999998e-05 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.324299+0000", 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036082750000000002 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.324386+0000", 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6541999999999999e-05 2024-09-14T07:08:02.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.283 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:02.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:02.285 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:02.282+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:02.385 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:02.382+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:02.472 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:02.472 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:02.486 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:02.482+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:02.487 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:02.587 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:02.582+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:02.663 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:02.663 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:02.663 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:02.663 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:02.663 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:02.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:02.687 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:02.686+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:02.788 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:02.786+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:7 3.1a e23/21 3:5cea7035:::rbd_info:head v 23'1, mlcod=0'0)", 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.007084939, 2024-09-14T07:08:02.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021634900000000001, 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798804+0000", 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798805+0000", 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1999999999999999e-07 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798819+0000", 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798879+0000", 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9373000000000002e-05 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.798902+0000", 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2836e-05 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.799063+0000", 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016089899999999999 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.799100+0000", 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7740000000000001e-05 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.800872+0000", 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017718390000000001 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:48.800967+0000", 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5292999999999997e-05 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:939 3.2 3:40fc327c:::rbd_data.11611b5ca054.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3473408~57344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.854240+0000", 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.9516492339999996, 2024-09-14T07:08:02.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012138079, 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 939 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854240+0000", 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854240+0000", 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854243+0000", 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.597e-06 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854526+0000", 2024-09-14T07:08:02.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000283038 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854570+0000", 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3726000000000001e-05 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854594+0000", 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3672999999999998e-05 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854779+0000", 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018488900000000001 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.855023+0000", 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024445800000000002 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.855488+0000", 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00046545199999999999 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865325+0000", 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098367479999999993 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866097+0000", 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00077223199999999997 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866258+0000", 2024-09-14T07:08:02.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160927 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866378+0000", 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000119339 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop_reply(client.4406.0:939 3.2 e26/21 ondisk, result = 0)", 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.860863+0000", 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.9450257050000008, 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044860810000000003, 2024-09-14T07:08:02.818 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.860863+0000", 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.860863+0000", 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.860866+0000", 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3659999999999999e-06 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.860878+0000", 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2476999999999999e-05 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.860913+0000", 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5405999999999999e-05 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.860933+0000", 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.986e-05 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865282+0000", 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043490209999999998 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865305+0000", 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3082999999999999e-05 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865349+0000", 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3868000000000001e-05 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:942 3.8 e26/25 3:1016b2ca:::rbd_data.11611b5ca054.0000000000000007:head v 26'3, mlcod=0'0)", 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.863071+0000", 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.9428179159999992, 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002479036, 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.822 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863071+0000", 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863071+0000", 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863073+0000", 2024-09-14T07:08:02.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.863e-06 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863151+0000", 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8129999999999999e-05 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863186+0000", 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5509000000000002e-05 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863206+0000", 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0007e-05 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863364+0000", 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000157367 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.863414+0000", 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0118999999999999e-05 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865465+0000", 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020513929999999999 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865550+0000", 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4648000000000007e-05 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1163 3.7 e26/25 3:e1672fbb:::rbd_data.11611b5ca054.000000000000001b:head v 26'1, mlcod=0'0)", 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.027599+0000", 2024-09-14T07:08:02.826 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7782897559999995, 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00268403, 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027599+0000", 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027599+0000", 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027601+0000", 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.294e-06 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027681+0000", 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9202000000000005e-05 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027714+0000", 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2966999999999998e-05 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027733+0000", 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.961e-05 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027907+0000", 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017386499999999999 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027961+0000", 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3625999999999998e-05 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.030175+0000", 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022145530000000002 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.030283+0000", 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010791299999999999 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1168 3.2 3:42220d69:::rbd_data.11611b5ca054.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3526656~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7718624950000006, 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014557588, 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1168 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034029+0000", 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9270000000000001e-06 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034081+0000", 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1313000000000002e-05 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034115+0000", 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4156000000000003e-05 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034131+0000", 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6350000000000001e-05 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034276+0000", 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014510900000000001 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034400+0000", 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012352899999999999 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034616+0000", 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021601199999999999 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.045899+0000", 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011283519000000001 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048374+0000", 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024744379999999998 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048466+0000", 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1952000000000003e-05 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048584+0000", 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011828300000000001 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1170 3.2 3:40d5ec5b:::rbd_data.11611b5ca054.000000000000002f:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4005888~61440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.034753+0000", 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7711359840000007, 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014288159999999999, 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.836 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1170 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034753+0000", 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034753+0000", 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034755+0000", 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.857e-06 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034813+0000", 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8686e-05 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034835+0000", 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1916000000000002e-05 2024-09-14T07:08:02.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034850+0000", 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4627000000000001e-05 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.037339+0000", 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024895239999999999 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.037555+0000", 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000215575 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.037781+0000", 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022602500000000001 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.046600+0000", 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0088193349999999993 2024-09-14T07:08:02.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048869+0000", 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022680930000000001 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048962+0000", 2024-09-14T07:08:02.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3721999999999995e-05 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.049041+0000", 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8800000000000004e-05 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1193 3.2 3:42220d69:::rbd_data.11611b5ca054.000000000000002d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.058574+0000", 2024-09-14T07:08:02.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7473149540000001, 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027492049999999998, 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1193 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058574+0000", 2024-09-14T07:08:02.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058574+0000", 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058576+0000", 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1900000000000002e-06 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058587+0000", 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1003e-05 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058619+0000", 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1846e-05 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058634+0000", 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4613000000000001e-05 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058763+0000", 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012961299999999999 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058910+0000", 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000147052 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.059080+0000", 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169999 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.060483+0000", 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001403262 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.061150+0000", 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00066631199999999996 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.061247+0000", 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7208999999999997e-05 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.061323+0000", 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6106000000000006e-05 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1194 3.2 3:40d5ec5b:::rbd_data.11611b5ca054.000000000000002f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.058668+0000", 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7472210789999991, 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0031058599999999998, 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1194 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058668+0000", 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058668+0000", 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058669+0000", 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3170000000000001e-06 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058677+0000", 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1529999999999997e-06 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058703+0000", 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6054000000000002e-05 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.058720+0000", 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6266e-05 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.059641+0000", 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00092172 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.059800+0000", 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000159142 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.059966+0000", 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000165852 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.061345+0000", 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001378556 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.061633+0000", 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028860099999999998 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.061709+0000", 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5915999999999996e-05 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.061774+0000", 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4282999999999994e-05 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1268 3.6 e26/21 3:6107e7ff:::rbd_data.11611b5ca054.0000000000000034:head v 26'3, mlcod=24'1)", 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.117265+0000", 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6886241440000003, 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028132949999999999, 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.854 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117265+0000", 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117265+0000", 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117266+0000", 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1699999999999997e-07 2024-09-14T07:08:02.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117354+0000", 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8164999999999994e-05 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117383+0000", 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8750000000000001e-05 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117395+0000", 2024-09-14T07:08:02.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2639999999999999e-05 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117545+0000", 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014939200000000001 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.117615+0000", 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0133000000000001e-05 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.119989+0000", 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023743620000000001 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.120078+0000", 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8936000000000004e-05 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1425 3.8 e26/25 3:1016b2ca:::rbd_data.11611b5ca054.0000000000000007:head v 26'4, mlcod=24'2)", 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.254386+0000", 2024-09-14T07:08:02.858 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5515026859999992, 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0026818089999999998, 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254386+0000", 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254386+0000", 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254388+0000", 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.08e-06 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254469+0000", 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0703000000000003e-05 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254498+0000", 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8566000000000001e-05 2024-09-14T07:08:02.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254517+0000", 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9199999999999999e-05 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254684+0000", 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000167002 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254746+0000", 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1890000000000005e-05 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.256978+0000", 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022321250000000002 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.257068+0000", 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0242999999999996e-05 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1589 3.7 e26/25 3:e1672fbb:::rbd_data.11611b5ca054.000000000000001b:head v 26'4, mlcod=26'2)", 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.353870+0000", 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.4520193300000006, 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094823960000000006, 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353870+0000", 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353870+0000", 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353871+0000", 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.933e-06 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353957+0000", 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5136000000000006e-05 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353986+0000", 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9543000000000002e-05 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354005+0000", 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8672999999999999e-05 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354133+0000", 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000128539 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354192+0000", 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-05 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.363246+0000", 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0090549930000000008 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.363352+0000", 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105419 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1590 3.7 e26/25 3:e1672fbb:::rbd_data.11611b5ca054.000000000000001b:head v 26'5, mlcod=26'2)", 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.361751+0000", 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.4441380450000008, 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024729079999999998, 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361751+0000", 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361751+0000", 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361753+0000", 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8139999999999999e-06 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361852+0000", 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9536000000000004e-05 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361881+0000", 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8543000000000001e-05 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361895+0000", 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3906e-05 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361991+0000", 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6545999999999993e-05 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.362026+0000", 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4656000000000002e-05 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.364148+0000", 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021223240000000001 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.364224+0000", 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5582999999999998e-05 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.871 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:626 3.2 3:4550e1e5:::rbd_id.image_client.0-clone1:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.897944+0000", 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.9079448660000002, 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005682983, 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 626 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.897944+0000", 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.897944+0000", 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.897947+0000", 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.897955+0000", 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.897971+0000", 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5668e-05 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.897984+0000", 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2755000000000001e-05 2024-09-14T07:08:02.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.898098+0000", 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000114259 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op must wait for map", 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.898112+0000", 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4433000000000001e-05 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.901284+0000", 2024-09-14T07:08:02.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0031719080000000002 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.901430+0000", 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014561199999999999 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.901626+0000", 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000195966 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.902589+0000", 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00096275600000000005 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.903522+0000", 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00093300299999999996 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.903589+0000", 2024-09-14T07:08:02.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7111000000000002e-05 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.903627+0000", 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8330000000000001e-05 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:629 3.19 3:9dc33942:::rbd_object_map.116147128396:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.908001+0000", 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.8978880480000004, 2024-09-14T07:08:02.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021262759999999999, 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 629 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908001+0000", 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908001+0000", 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908002+0000", 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6610000000000001e-06 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908009+0000", 2024-09-14T07:08:02.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0049999999999998e-06 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908026+0000", 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6141e-05 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908037+0000", 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1565e-05 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908101+0000", 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4096000000000004e-05 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908258+0000", 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015714799999999999 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.908443+0000", 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018507800000000001 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.909173+0000", 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072959399999999997 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.910017+0000", 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084381300000000001 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.910079+0000", 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2467999999999999e-05 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.910127+0000", 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7707000000000002e-05 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4539.0:2029 3.b 3:d71b040a:::rbd_object_map.116147128396.0000000000000004:head [writefull 0~38] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:57.763408+0000", 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.0424806240000004, 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0038579080000000002, 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4539", 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2491585557", 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2029 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763408+0000", 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763408+0000", 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763410+0000", 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7740000000000001e-06 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763419+0000", 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7860000000000002e-06 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763440+0000", 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1503999999999999e-05 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763451+0000", 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1195999999999999e-05 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763585+0000", 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013331899999999999 2024-09-14T07:08:02.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.763940+0000", 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035475399999999999 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.764323+0000", 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00038320699999999999 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.766329+0000", 2024-09-14T07:08:02.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020066319999999999 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767123+0000", 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079346699999999996 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767204+0000", 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0785999999999995e-05 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767266+0000", 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2483000000000005e-05 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4539.0:2030 3.19 3:9dc33942:::rbd_object_map.116147128396:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:57.767698+0000", 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.0381903570000004, 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040456329999999999, 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4539", 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2491585557", 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2030 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767698+0000", 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767698+0000", 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767700+0000", 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9570000000000001e-06 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767711+0000", 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0623e-05 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767739+0000", 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8144e-05 2024-09-14T07:08:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767753+0000", 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3923e-05 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.767899+0000", 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000146002 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.768169+0000", 2024-09-14T07:08:02.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026959800000000002 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.768808+0000", 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063884200000000003 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.770373+0000", 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001565774 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.771518+0000", 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011451020000000001 2024-09-14T07:08:02.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.771674+0000", 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000155525 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.771744+0000", 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0142999999999996e-05 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.895 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1089 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:57.818698+0000", 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9871906209999999, 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034111279999999998, 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1089 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.818698+0000", 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.818698+0000", 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.818700+0000", 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.818712+0000", 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1093e-05 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.818744+0000", 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2252999999999998e-05 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.818759+0000", 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.818883+0000", 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012406900000000001 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.819098+0000", 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021534200000000001 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.819395+0000", 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00029710099999999997 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.821469+0000", 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020737310000000001 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.821969+0000", 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00050044900000000003 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.822049+0000", 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9880000000000001e-05 2024-09-14T07:08:02.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:57.822109+0000", 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0056000000000002e-05 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4539.0:2106 3.19 3:9dc33942:::rbd_object_map.116147128396:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:58.094073+0000", 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7118159410000002, 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003396395, 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4539", 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2491585557", 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2106 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094073+0000", 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094073+0000", 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094074+0000", 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.576e-06 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094084+0000", 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094104+0000", 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0641000000000001e-05 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094114+0000", 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8509999999999994e-06 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094277+0000", 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016282800000000001 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.094537+0000", 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025977800000000001 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.095046+0000", 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00050886599999999996 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.096217+0000", 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001171861 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.097341+0000", 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011232239999999999 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.097411+0000", 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0300000000000001e-05 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.097469+0000", 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8285999999999997e-05 2024-09-14T07:08:02.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4507.0:327 3.2 3:40893f22:::rbd_data.116147128396.0000000000000004:head [read 1572864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:01.907646+0000", 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.89824269000000001, 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022364500000000001, 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:02.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4507", 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/682530424", 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 327 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907646+0000", 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907646+0000", 2024-09-14T07:08:02.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907649+0000", 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5859999999999999e-06 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907657+0000", 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:02.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907682+0000", 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5568999999999999e-05 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907693+0000", 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0712e-05 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907762+0000", 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9214000000000005e-05 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op must wait for map", 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.907767+0000", 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.228e-06 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.909729+0000", 2024-09-14T07:08:02.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019626859999999999 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.909806+0000", 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7276999999999999e-05 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:01.909883+0000", 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6113999999999997e-05 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:02.914 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:02.914 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:02.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:02.916 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:02.914+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:03.016 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:03.014+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:03.092 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:03.092 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:03.092 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:03.093 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:03.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:03.151 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:03.150+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:03.166 INFO:teuthology.orchestra.run.smithi145.stdout:486 read 0x7010e00 thru 0x701b558 (0xa759 bytes) 2024-09-14T07:08:03.167 INFO:teuthology.orchestra.run.smithi145.stdout:488 write 0xa2d800 thru 0xa301a2 (0x29a3 bytes) 2024-09-14T07:08:03.169 INFO:teuthology.orchestra.run.smithi145.stdout:489 write 0x51e4200 thru 0x51ee513 (0xa314 bytes) 2024-09-14T07:08:03.170 INFO:teuthology.orchestra.run.smithi145.stdout:490 clone 4 order 19 su 131072 sc 2 2024-09-14T07:08:03.218 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:03.214+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:03.236 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:03.237 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:03.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:03.343 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:03.342+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:03.419 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:03.418+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:928 3.5 e26/25 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head v 26'6, mlcod=0'0)", 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.846486+0000", 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5743315360000008, 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010428850999999999, 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846486+0000", 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846486+0000", 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846488+0000", 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9630000000000001e-06 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846547+0000", 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9048000000000001e-05 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846579+0000", 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2233999999999997e-05 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846601+0000", 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.126e-05 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846746+0000", 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014580100000000001 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846801+0000", 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4543000000000003e-05 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.856784+0000", 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0099830330000000005 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.856915+0000", 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013096900000000001 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:929 3.9 e26/25 3:92b5a216:::rbd_object_map.11611b5ca054:head v 26'85, mlcod=24'83)", 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.846971+0000", 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5738467020000009, 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036026159999999999, 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846971+0000", 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846971+0000", 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846973+0000", 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.72e-06 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.846989+0000", 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5888000000000001e-05 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.847009+0000", 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0307999999999999e-05 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.847020+0000", 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1306e-05 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.847133+0000", 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011304800000000001 2024-09-14T07:08:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.847199+0000", 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6175999999999998e-05 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.850467+0000", 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003267261 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.850574+0000", 2024-09-14T07:08:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106909 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:931 3.9 e26/25 3:92b5a216:::rbd_object_map.11611b5ca054:head v 26'87, mlcod=24'83)", 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.848272+0000", 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5725460180000006, 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003623102, 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.848272+0000", 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.848272+0000", 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.848280+0000", 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0539999999999995e-06 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.848290+0000", 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0176999999999999e-05 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.848312+0000", 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2393999999999999e-05 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.848337+0000", 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4445999999999999e-05 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849432+0000", 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010956469999999999 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849503+0000", 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0426000000000006e-05 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.851777+0000", 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022738559999999999 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.851895+0000", 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118102 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:932 3.9 e26/25 3:92b5a216:::rbd_object_map.11611b5ca054:head v 26'88, mlcod=24'83)", 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.849637+0000", 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5711806189999997, 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035376370000000002, 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849637+0000", 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849637+0000", 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849639+0000", 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.934e-06 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849653+0000", 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4e-05 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849677+0000", 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3853e-05 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.849689+0000", 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2407e-05 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.850800+0000", 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011112509999999999 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.850897+0000", 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6490000000000006e-05 2024-09-14T07:08:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853061+0000", 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021637890000000002 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853175+0000", 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011391300000000001 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:935 3.9 e26/25 3:92b5a216:::rbd_object_map.11611b5ca054:head v 26'90, mlcod=24'83)", 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.853120+0000", 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5676979250000009, 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0042881229999999996, 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853120+0000", 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853120+0000", 2024-09-14T07:08:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853122+0000", 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9e-06 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853136+0000", 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4017e-05 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853160+0000", 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4303000000000001e-05 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853178+0000", 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8428999999999999e-05 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853308+0000", 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012972299999999999 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.853366+0000", 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7676000000000001e-05 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.857291+0000", 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0039255389999999996 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.857408+0000", 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116536 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:942 3.8 3:1016b2ca:::rbd_data.11611b5ca054.0000000000000007:head [set-alloc-hint object_size 4194304 write_size 4194304,write 13824~25088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.861902+0000", 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5589160129999993, 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045150449999999996, 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 942 2024-09-14T07:08:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.861902+0000", 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.861902+0000", 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.861904+0000", 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8100000000000002e-06 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.862021+0000", 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116809 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.862069+0000", 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7673000000000002e-05 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.862085+0000", 2024-09-14T07:08:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5690000000000001e-05 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.862214+0000", 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000129281 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.862617+0000", 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00040336799999999998 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.862872+0000", 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000254551 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865709+0000", 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028368880000000001 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866170+0000", 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000461211 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866266+0000", 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6149000000000004e-05 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866417+0000", 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015061499999999999 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1153 3.18 3:1f883cd2:::rbd_data.11611b5ca054.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write 802816~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.024429+0000", 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3963891410000002, 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01468203, 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1153 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.024429+0000", 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.024429+0000", 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.024432+0000", 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.393e-06 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.024698+0000", 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000266302 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.024771+0000", 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2622999999999999e-05 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.024798+0000", 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6859000000000002e-05 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.024950+0000", 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000152652 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025170+0000", 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021994500000000001 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025407+0000", 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023661900000000001 2024-09-14T07:08:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.037514+0000", 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012107503 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038939+0000", 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014249919999999999 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.039036+0000", 2024-09-14T07:08:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6842999999999994e-05 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.039111+0000", 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4299000000000001e-05 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1163 3.7 3:e1672fbb:::rbd_data.11611b5ca054.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write 565248~45056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.026289+0000", 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3945288619999996, 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046873740000000002, 2024-09-14T07:08:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1163 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026289+0000", 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026289+0000", 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026291+0000", 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.08e-06 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026413+0000", 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012214600000000001 2024-09-14T07:08:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026444+0000", 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1442999999999997e-05 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026461+0000", 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6096e-05 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026628+0000", 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016725600000000001 2024-09-14T07:08:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026992+0000", 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00036430000000000002 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.027387+0000", 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00039509700000000002 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.030508+0000", 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003120823 2024-09-14T07:08:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.030671+0000", 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016332900000000001 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.030838+0000", 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016646900000000001 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.030976+0000", 2024-09-14T07:08:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013833500000000001 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1158 3.d e26/25 3:b2d0cb95:::rbd_data.11611b5ca054.0000000000000022:head v 26'1, mlcod=0'0)", 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.026481+0000", 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3943371370000008, 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011840572000000001, 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026481+0000", 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026481+0000", 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026483+0000", 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026603+0000", 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120419 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026638+0000", 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5339999999999997e-05 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026666+0000", 2024-09-14T07:08:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.724e-05 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026797+0000", 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013174899999999999 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.026860+0000", 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2848999999999994e-05 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038236+0000", 2024-09-14T07:08:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011376156 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038321+0000", 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4652000000000002e-05 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1354 3.d e26/25 3:b2d0cb95:::rbd_data.11611b5ca054.0000000000000022:head v 26'2, mlcod=0'0)", 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.181572+0000", 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2392456109999994, 2024-09-14T07:08:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0073426840000000004, 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181572+0000", 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181572+0000", 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181573+0000", 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9400000000000004e-07 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181638+0000", 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5038999999999999e-05 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181652+0000", 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4367e-05 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181661+0000", 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.003e-06 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181828+0000", 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016630200000000001 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.181866+0000", 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8819999999999998e-05 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.188728+0000", 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0068618699999999999 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.188915+0000", 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018638899999999999 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1425 3.8 3:1016b2ca:::rbd_data.11611b5ca054.0000000000000007:head [write 4055040~51712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.253041+0000", 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1677768349999997, 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046235010000000003, 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1425 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253041+0000", 2024-09-14T07:08:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253041+0000", 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253043+0000", 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.193e-06 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253092+0000", 2024-09-14T07:08:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9073000000000003e-05 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253188+0000", 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5976000000000004e-05 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253206+0000", 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8090000000000001e-05 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253370+0000", 2024-09-14T07:08:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016413899999999999 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.253640+0000", 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026923099999999998 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.254161+0000", 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00052132600000000002 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.256672+0000", 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0025107509999999999 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.257353+0000", 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00068149699999999998 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.257513+0000", 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015970599999999999 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.257664+0000", 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000151519 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1443 3.7 3:e1672fbb:::rbd_data.11611b5ca054.000000000000001b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.265849+0000", 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1549690169999991, 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034286, 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1443 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.265849+0000", 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.265849+0000", 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.265851+0000", 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1459999999999999e-06 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.265861+0000", 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.06e-05 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.265893+0000", 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1683000000000001e-05 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.265909+0000", 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6426000000000002e-05 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.266062+0000", 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015246300000000001 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.266322+0000", 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025960399999999998 2024-09-14T07:08:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.266667+0000", 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000345951 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.267944+0000", 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012763469999999999 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.268984+0000", 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010398460000000001 2024-09-14T07:08:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.269145+0000", 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000161552 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.269277+0000", 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000131982 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1589 3.7 3:e1672fbb:::rbd_data.11611b5ca054.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4145152~49152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.353044+0000", 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0677740139999994, 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010849599999999999, 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1589 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353044+0000", 2024-09-14T07:08:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353044+0000", 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353046+0000", 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4169999999999999e-06 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353103+0000", 2024-09-14T07:08:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6483000000000001e-05 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353134+0000", 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1519000000000001e-05 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353149+0000", 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5299999999999999e-05 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353281+0000", 2024-09-14T07:08:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013168300000000001 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353437+0000", 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000155892 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353695+0000", 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025801099999999999 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.363288+0000", 2024-09-14T07:08:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095927259999999993 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.363696+0000", 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00040868399999999998 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.363810+0000", 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113292 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.363893+0000", 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3592999999999997e-05 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1590 3.7 3:e1672fbb:::rbd_data.11611b5ca054.000000000000001b:head [set-alloc-hint object_size 4194304 write_size 4194304,write 601088~59392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.353199+0000", 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0676182179999998, 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011426255, 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1590 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353199+0000", 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353199+0000", 2024-09-14T07:08:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353201+0000", 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353265+0000", 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3756000000000004e-05 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353284+0000", 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9516000000000001e-05 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.353295+0000", 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0874e-05 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.360957+0000", 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0076616139999999998 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361148+0000", 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019116099999999999 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.361471+0000", 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000323448 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.364272+0000", 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028007990000000001 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.364480+0000", 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000207808 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.364564+0000", 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3925999999999996e-05 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.364626+0000", 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1902999999999993e-05 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1592 3.18 3:1f883cd2:::rbd_data.11611b5ca054.000000000000001c:head [set-alloc-hint object_size 4194304 write_size 4194304,write 0~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.354761+0000", 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.06605667, 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035758750000000001, 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1592 2024-09-14T07:08:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354761+0000", 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354761+0000", 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354763+0000", 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3630000000000001e-06 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354794+0000", 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0756000000000002e-05 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354833+0000", 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9257e-05 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354859+0000", 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5225999999999999e-05 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.354984+0000", 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000125346 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.355123+0000", 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013894600000000001 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.355354+0000", 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023141399999999999 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.358103+0000", 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002749016 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.358179+0000", 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5401999999999994e-05 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.358272+0000", 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3063e-05 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.358337+0000", 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5086000000000007e-05 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1729 3.1e 3:78dd04c6:::rbd_data.11611b5ca054.0000000000000039:head [set-alloc-hint object_size 4194304 write_size 4194304,write 479232~24576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.462450+0000", 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.9583679150000002, 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034807499999999999, 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1729 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462450+0000", 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462450+0000", 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462452+0000", 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.277e-06 2024-09-14T07:08:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462485+0000", 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3408999999999998e-05 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462517+0000", 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1717000000000001e-05 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462533+0000", 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5716999999999999e-05 2024-09-14T07:08:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462687+0000", 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153862 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.462947+0000", 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000260005 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.463186+0000", 2024-09-14T07:08:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023951800000000001 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.464849+0000", 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016625329999999999 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.465618+0000", 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00076886399999999996 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.465803+0000", 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000185422 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.465930+0000", 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012742600000000001 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4471.0:786 3.1e 3:7b4838f8:::rbd_data.11611b5ca054.0000000000000016:head [sparse-read 135168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:58.853226+0000", 2024-09-14T07:08:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.567591814, 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053073410000000001, 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4471", 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1388298560", 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 786 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853226+0000", 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853226+0000", 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853227+0000", 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.269e-06 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853233+0000", 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5999999999999997e-06 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853252+0000", 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9219999999999999e-05 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853261+0000", 2024-09-14T07:08:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2650000000000002e-06 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853345+0000", 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3949000000000005e-05 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op must wait for map", 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853353+0000", 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.028e-06 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.858384+0000", 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0050312439999999998 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.858533+0000", 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014876599999999999 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4471.0:787 3.1e 3:7b4838f8:::rbd_data.11611b5ca054.0000000000000016:head [sparse-read 266240~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:58.853421+0000", 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.5673961700000003, 2024-09-14T07:08:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0052533850000000002, 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4471", 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1388298560", 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 787 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853421+0000", 2024-09-14T07:08:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853421+0000", 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853423+0000", 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0559999999999999e-06 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853428+0000", 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4949999999999999e-06 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853438+0000", 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0241000000000001e-05 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853447+0000", 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5259999999999993e-06 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853684+0000", 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023672 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting_for_map not empty", 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.853709+0000", 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.529e-05 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.858561+0000", 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0048523409999999996 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:58.858675+0000", 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011371599999999999 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:203 3.1e 3:7c24cf2c:::rbd_data.116148d60084.0000000000000017:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:03.184720+0000", 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.236097472, 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034525839999999999, 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 203 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.184720+0000", 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.184720+0000", 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.184721+0000", 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3260000000000001e-06 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.184729+0000", 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3039999999999999e-06 2024-09-14T07:08:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.184746+0000", 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7592999999999999e-05 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.184756+0000", 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4979999999999995e-06 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.184845+0000", 2024-09-14T07:08:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9580999999999995e-05 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185070+0000", 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022441900000000001 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185383+0000", 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000313092 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.186558+0000", 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011753969999999999 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.188065+0000", 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015065370000000001 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.188118+0000", 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2927999999999999e-05 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.188173+0000", 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4908999999999999e-05 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4498.0:201 3.5 e33/25 3:a6f34506:::rbd_data.116148d60084.00000000000000a0:head v 33'14, mlcod=31'12)", 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:03.185141+0000", 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.23567666500000001, 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003510558, 2024-09-14T07:08:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185141+0000", 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185141+0000", 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185142+0000", 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6899999999999996e-07 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185319+0000", 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017713399999999999 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185329+0000", 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0172e-05 2024-09-14T07:08:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185335+0000", 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2650000000000002e-06 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185406+0000", 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0914000000000006e-05 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.185426+0000", 2024-09-14T07:08:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9507000000000001e-05 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.188536+0000", 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0031101670000000001 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:03.188652+0000", 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011563 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:03.523 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:03.524 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:03.524 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:03.522+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:03.626 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:03.622+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:03.657 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:03.657 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:03.658 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:03.658 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:03.671 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:03.726 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:03.726+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:03.827 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:03.826+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:03.841 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:03.841 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:03.855 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:03.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:08:03.929 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:03.926+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:03.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:03.981 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.677628864999999, 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:20 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.643503+0000", 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.334629721999999, 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012546646, 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 20 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643503+0000", 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643503+0000", 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643506+0000", 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.627e-06 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643680+0000", 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017461500000000001 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643712+0000", 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2172999999999998e-05 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643730+0000", 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7110000000000001e-05 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643880+0000", 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000150929 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644102+0000", 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022136399999999999 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644478+0000", 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037612700000000003 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651054+0000", 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065762340000000002 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.655704+0000", 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046501169999999996 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.655897+0000", 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019290799999999999 2024-09-14T07:08:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656050+0000", 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000152442 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:21 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 65536~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.643770+0000", 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.334362614, 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012495286, 2024-09-14T07:08:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643770+0000", 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643770+0000", 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643771+0000", 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3170000000000001e-06 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643927+0000", 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015602500000000001 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643944+0000", 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6745999999999999e-05 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643955+0000", 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0377000000000001e-05 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.646417+0000", 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024623679999999999 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.646653+0000", 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00023595199999999999 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.647026+0000", 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037282999999999999 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.655664+0000", 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0086378099999999992 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656110+0000", 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044634600000000003 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656190+0000", 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9789999999999993e-05 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656265+0000", 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5724999999999997e-05 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:03.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.643981+0000", 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.334151639000002, 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012545588999999999, 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643981+0000", 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643981+0000", 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.643982+0000", 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1030000000000001e-06 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644058+0000", 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5346000000000007e-05 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644071+0000", 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3436e-05 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644081+0000", 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5670000000000001e-06 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.648660+0000", 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045795419999999998 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.648948+0000", 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000288351 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.649331+0000", 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038227399999999999 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656299+0000", 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0069682520000000003 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656391+0000", 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2120000000000006e-05 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656469+0000", 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8026000000000001e-05 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656527+0000", 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7571999999999997e-05 2024-09-14T07:08:04.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:23 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.644098+0000", 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.334034636999998, 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012692985, 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 23 2024-09-14T07:08:04.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644098+0000", 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644098+0000", 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644099+0000", 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6700000000000002e-07 2024-09-14T07:08:04.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644240+0000", 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000140572 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644253+0000", 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3793e-05 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644265+0000", 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1754e-05 2024-09-14T07:08:04.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651272+0000", 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070070089999999998 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651460+0000", 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000188106 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.651673+0000", 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000212258 2024-09-14T07:08:04.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656639+0000", 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0049660110000000002 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656669+0000", 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0766000000000003e-05 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656736+0000", 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6509999999999998e-05 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656791+0000", 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5238999999999997e-05 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:24 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.644284+0000", 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.333849221000001, 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013588008, 2024-09-14T07:08:04.011 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 24 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644284+0000", 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644284+0000", 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644285+0000", 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1230000000000001e-06 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644428+0000", 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014294500000000001 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644441+0000", 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3893000000000001e-05 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644451+0000", 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5040000000000008e-06 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.653354+0000", 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0089025779999999995 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.653581+0000", 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022751499999999999 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.653970+0000", 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038934000000000001 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656824+0000", 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028534979999999999 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657727+0000", 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00090278999999999999 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657808+0000", 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1719999999999997e-05 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657872+0000", 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3102000000000007e-05 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.644470+0000", 2024-09-14T07:08:04.017 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.333662619999998, 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014639968, 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 25 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.018 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644470+0000", 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644470+0000", 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644471+0000", 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5999999999999991e-07 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644478+0000", 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6329999999999999e-06 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644488+0000", 2024-09-14T07:08:04.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0349999999999999e-05 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.644496+0000", 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6000000000000001e-06 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.656922+0000", 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012426368 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657054+0000", 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013235499999999999 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.657266+0000", 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021203300000000001 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.658289+0000", 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010222720000000001 2024-09-14T07:08:04.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.658983+0000", 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00069388100000000003 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.659057+0000", 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4553000000000003e-05 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.659110+0000", 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2963000000000001e-05 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.754783+0000", 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.223349560999999, 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011459977999999999, 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.024 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 43 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754783+0000", 2024-09-14T07:08:04.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754783+0000", 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754784+0000", 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0899999999999999e-06 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754791+0000", 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754803+0000", 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.241e-05 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754815+0000", 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1357e-05 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.761307+0000", 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064925 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.761418+0000", 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110299 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.761589+0000", 2024-09-14T07:08:04.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000171025 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.765799+0000", 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042101129999999997 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.765976+0000", 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017721999999999999 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766124+0000", 2024-09-14T07:08:04.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000148138 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766243+0000", 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119076 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.754969+0000", 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.223163974999999, 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011519387000000001, 2024-09-14T07:08:04.030 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 44 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754969+0000", 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754969+0000", 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.754970+0000", 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755047+0000", 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7245999999999999e-05 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755062+0000", 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4929000000000001e-05 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755075+0000", 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.325e-05 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.762648+0000", 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0075726860000000003 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.762760+0000", 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112248 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.762938+0000", 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017746400000000001 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.765932+0000", 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029937050000000002 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766266+0000", 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033474999999999998 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766390+0000", 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012400000000000001 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766488+0000", 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7829000000000001e-05 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755165+0000", 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222967384, 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012304374999999999, 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 45 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.037 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755165+0000", 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755165+0000", 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755167+0000", 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:08:04.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755176+0000", 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0170000000000002e-06 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755191+0000", 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4932999999999999e-05 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755200+0000", 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3700000000000001e-06 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.764320+0000", 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0091199490000000005 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.764550+0000", 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022994199999999999 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.764941+0000", 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00039086399999999999 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767179+0000", 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022382320000000002 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767216+0000", 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166000000000003e-05 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767345+0000", 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000129176 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767470+0000", 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124509 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755218+0000", 2024-09-14T07:08:04.042 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222914744000001, 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016364961000000001, 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 46 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755218+0000", 2024-09-14T07:08:04.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755218+0000", 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755219+0000", 2024-09-14T07:08:04.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1999999999999998e-07 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755249+0000", 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9943000000000001e-05 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755261+0000", 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1987000000000001e-05 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755269+0000", 2024-09-14T07:08:04.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.123e-06 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766556+0000", 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011286608 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766622+0000", 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6699000000000006e-05 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.766729+0000", 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010721999999999999 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771118+0000", 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043881969999999999 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771333+0000", 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000215672 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771468+0000", 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013462599999999999 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771583+0000", 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114966 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.048 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755285+0000", 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222847863999998, 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016543782, 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.049 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 47 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755285+0000", 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755285+0000", 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755286+0000", 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0359999999999999e-06 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755389+0000", 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103286 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755415+0000", 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6282999999999999e-05 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755424+0000", 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0340000000000002e-06 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767674+0000", 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012249711 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.767816+0000", 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014178899999999999 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.768030+0000", 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021378200000000001 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771292+0000", 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032618479999999999 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771608+0000", 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031678799999999998 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771735+0000", 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012682899999999999 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771829+0000", 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3395999999999998e-05 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755459+0000", 2024-09-14T07:08:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222673383, 2024-09-14T07:08:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017949085, 2024-09-14T07:08:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.056 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.056 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.056 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.056 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 48 2024-09-14T07:08:04.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.056 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755459+0000", 2024-09-14T07:08:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755459+0000", 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755461+0000", 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.184e-06 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755469+0000", 2024-09-14T07:08:04.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9460000000000004e-06 2024-09-14T07:08:04.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755483+0000", 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3213999999999999e-05 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755491+0000", 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0099999999999995e-06 2024-09-14T07:08:04.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.769464+0000", 2024-09-14T07:08:04.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013973785000000001 2024-09-14T07:08:04.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.769707+0000", 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024299799999999999 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.770118+0000", 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041058700000000002 2024-09-14T07:08:04.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773131+0000", 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030129940000000002 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773167+0000", 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6288999999999998e-05 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773300+0000", 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013227599999999999 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773408+0000", 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010880200000000001 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222625152999999, 2024-09-14T07:08:04.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022538355999999999, 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 49 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:08:04.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755508+0000", 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6400000000000001e-07 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755514+0000", 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2800000000000003e-06 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755527+0000", 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.341e-05 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755536+0000", 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:08:04.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.771932+0000", 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016396852999999999 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.772068+0000", 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013607300000000001 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.772298+0000", 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000229528 2024-09-14T07:08:04.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.776449+0000", 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041507469999999998 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.777782+0000", 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013336839999999999 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.777921+0000", 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013824900000000001 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778046+0000", 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000125282 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755548+0000", 2024-09-14T07:08:04.070 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222584569999999, 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022670177, 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755548+0000", 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755548+0000", 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755549+0000", 2024-09-14T07:08:04.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1699999999999997e-07 2024-09-14T07:08:04.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755644+0000", 2024-09-14T07:08:04.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4913000000000004e-05 2024-09-14T07:08:04.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755664+0000", 2024-09-14T07:08:04.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0259999999999999e-05 2024-09-14T07:08:04.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755678+0000", 2024-09-14T07:08:04.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3949e-05 2024-09-14T07:08:04.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773598+0000", 2024-09-14T07:08:04.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017920373999999999 2024-09-14T07:08:04.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773773+0000", 2024-09-14T07:08:04.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017479600000000001 2024-09-14T07:08:04.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.773990+0000", 2024-09-14T07:08:04.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021722700000000001 2024-09-14T07:08:04.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778071+0000", 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0040802160000000002 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778116+0000", 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5216999999999998e-05 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778162+0000", 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5775999999999997e-05 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778218+0000", 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6632000000000003e-05 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.079 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755705+0000", 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222427937999999, 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022896933000000001, 2024-09-14T07:08:04.080 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 51 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.081 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755705+0000", 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755705+0000", 2024-09-14T07:08:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755706+0000", 2024-09-14T07:08:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5239999999999999e-06 2024-09-14T07:08:04.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755718+0000", 2024-09-14T07:08:04.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1513e-05 2024-09-14T07:08:04.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755736+0000", 2024-09-14T07:08:04.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8199999999999999e-05 2024-09-14T07:08:04.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755750+0000", 2024-09-14T07:08:04.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3986e-05 2024-09-14T07:08:04.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.774878+0000", 2024-09-14T07:08:04.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.019128432000000001 2024-09-14T07:08:04.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.775122+0000", 2024-09-14T07:08:04.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024310799999999999 2024-09-14T07:08:04.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.775531+0000", 2024-09-14T07:08:04.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00040984099999999998 2024-09-14T07:08:04.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778327+0000", 2024-09-14T07:08:04.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027959479999999999 2024-09-14T07:08:04.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778363+0000", 2024-09-14T07:08:04.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5790000000000001e-05 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778495+0000", 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131409 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778602+0000", 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107182 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.091 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.755801+0000", 2024-09-14T07:08:04.092 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.222331655000001, 2024-09-14T07:08:04.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.023029786, 2024-09-14T07:08:04.092 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.092 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.092 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 52 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755801+0000", 2024-09-14T07:08:04.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755801+0000", 2024-09-14T07:08:04.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755802+0000", 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.367e-06 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755813+0000", 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.059e-05 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755841+0000", 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7662e-05 2024-09-14T07:08:04.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.755858+0000", 2024-09-14T07:08:04.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7640000000000001e-05 2024-09-14T07:08:04.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.776554+0000", 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.020695344000000001 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.776805+0000", 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025167100000000002 2024-09-14T07:08:04.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.777185+0000", 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037977699999999998 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778627+0000", 2024-09-14T07:08:04.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001441559 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778736+0000", 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108942 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778779+0000", 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2926999999999997e-05 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.778831+0000", 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2306999999999999e-05 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.958152+0000", 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.019980504999999, 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011192554, 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 75 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958152+0000", 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958152+0000", 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958153+0000", 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.046e-06 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958261+0000", 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958276+0000", 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4656e-05 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958286+0000", 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4809999999999995e-06 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.966282+0000", 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0079967140000000003 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.966345+0000", 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3138000000000004e-05 2024-09-14T07:08:04.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.966442+0000", 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6281999999999997e-05 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967983+0000", 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015415240000000001 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969293+0000", 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00130934 2024-09-14T07:08:04.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969323+0000", 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0306000000000001e-05 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969345+0000", 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.192e-05 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.106 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.958307+0000", 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.019825836999999, 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011418697, 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 76 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958307+0000", 2024-09-14T07:08:04.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958307+0000", 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958308+0000", 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6600000000000005e-07 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958313+0000", 2024-09-14T07:08:04.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5419999999999996e-06 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958322+0000", 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.958354+0000", 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2017999999999997e-05 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967384+0000", 2024-09-14T07:08:04.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.009029825 2024-09-14T07:08:04.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967440+0000", 2024-09-14T07:08:04.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6264999999999997e-05 2024-09-14T07:08:04.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:04.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.967541+0000", 2024-09-14T07:08:04.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100713 2024-09-14T07:08:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.968204+0000", 2024-09-14T07:08:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066269600000000005 2024-09-14T07:08:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969666+0000", 2024-09-14T07:08:04.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001462472 2024-09-14T07:08:04.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969696+0000", 2024-09-14T07:08:04.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9703999999999999e-05 2024-09-14T07:08:04.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.969726+0000", 2024-09-14T07:08:04.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9759000000000001e-05 2024-09-14T07:08:04.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.115 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.115 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1158 3.d 3:b2d0cb95:::rbd_data.11611b5ca054.0000000000000022:head [set-alloc-hint object_size 4194304 write_size 4194304,write 888832~49152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:04.115 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024683+0000", 2024-09-14T07:08:04.115 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.9534493200000007, 2024-09-14T07:08:04.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0133978, 2024-09-14T07:08:04.116 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:04.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:04.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:04.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:04.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:04.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1158 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024683+0000", 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:04.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024683+0000", 2024-09-14T07:08:04.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:04.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:04.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024685+0000", 2024-09-14T07:08:04.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9700000000000002e-06 2024-09-14T07:08:04.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:04.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024871+0000", 2024-09-14T07:08:04.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018548100000000001 2024-09-14T07:08:04.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:04.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024900+0000", 2024-09-14T07:08:04.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9547e-05 2024-09-14T07:08:04.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:04.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024915+0000", 2024-09-14T07:08:04.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4388e-05 2024-09-14T07:08:04.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:04.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025046+0000", 2024-09-14T07:08:04.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013156100000000001 2024-09-14T07:08:04.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025216+0000", 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017013 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025401+0000", 2024-09-14T07:08:04.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018443800000000001 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.031328+0000", 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0059274949999999996 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037954+0000", 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066254369999999996 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.038037+0000", 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3038e-05 2024-09-14T07:08:04.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:04.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:04.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:04.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.038081+0000", 2024-09-14T07:08:04.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4314999999999998e-05 2024-09-14T07:08:04.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.126 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:04.127 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:04.127 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:04.128 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:04.129 INFO:teuthology.orchestra.run.smithi145.stdout:flattening image image_client.0-clone3 2024-09-14T07:08:04.130 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:04.130 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:04.126+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:04.131 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:04.139 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:04.139 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:04.157 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:04.185 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:04.186 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:04.210 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:04.228 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:04.229 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:04.226+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:04.308 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:04.308 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:04.322 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:04.329 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:04.326+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:04.431 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:04.430+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:04.486 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:04.486 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:04.501 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:04.551 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:04.550+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:04.627 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:04.628 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:04.632 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:04.630+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:04.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:04.720 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T07:08:04.734 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:04.730+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:04.812 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:04.812 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:04.826 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:04.835 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:04.834+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:04.890 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:08:04.893 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:04.894 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:04.894 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T07:08:04.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:08:04.936 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:04.934+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:05.001 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:05.001 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:05.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:05.059 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:05.058+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:05.136 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:05.134+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:05.194 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:05.194 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:05.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:05.237 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:05.234+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:05.240 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:08:05.240 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:05.240 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 35, 2024-09-14T07:08:05.240 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:08:04.889212+0000", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:07:52.806531+0000", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5804032, 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:05.241 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:08:05.242 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:08:05.243 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:08:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.246 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:08:05.247 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 6.059999942779541, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 6.059999942779541, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.33000001311302185, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-09-14T07:08:05.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:08:05.249 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:08:05.250 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:08:05.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:08:05.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:08:05.253 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 2.25, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2.25, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-09-14T07:08:05.254 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:08:05.255 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:08:05.256 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "34", 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 6, 2024-09-14T07:08:05.257 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 34, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:08:05.258 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:08:05.259 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:08:05.260 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.5, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.5, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.5, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.5, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 21, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:05.262 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.263 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:05.265 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.266 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:05.267 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:08:05.268 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.269 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:08:05.270 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:05.272 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:08:05.273 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:05.275 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.275 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:08:05.275 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:05.276 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.278 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.278 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:05.278 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:08:05.279 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:08:05.280 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:05.281 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:05.282 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:05.282 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:05.282 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:05.282 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:05.282 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:08:05.282 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.284 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.285 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:08:05.286 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.287 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.292 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 25, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:05.298 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:05.300 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:05.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:05.302 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:08:05.303 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:08:05.304 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:08:05.305 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:08:05.305 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:08:05.305 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:05.305 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:05.307 INFO:tasks.thrashosds.thrasher:j is {'epoch': 35, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:08:04.889212+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:07:52.806531+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 6, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 21, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:08:05.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:08:05.346 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:05.347 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:05.347 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:05.346+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:05.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:05.438 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:05.434+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:05.532 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:05.532 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:05.539 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:05.538+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:05.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:05.640 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:05.638+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:05.664 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:05.664 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:05.673 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:08:05.674 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:08:05.676 INFO:teuthology.orchestra.run.smithi145.stdout:491 read 0x4731600 thru 0x473e42e (0xce2f bytes) 2024-09-14T07:08:05.678 INFO:teuthology.orchestra.run.smithi145.stdout:492 punch from 0x5eb7400 to 0x5eb9600, (0x2200 bytes) 2024-09-14T07:08:05.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:05.688 INFO:teuthology.orchestra.run.smithi145.stdout:493 read 0x1c75e00 thru 0x1c82a2e (0xcc2f bytes) 2024-09-14T07:08:05.695 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-14T07:08:05.212023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121958840,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":9147,"num_read_kb":15505,"num_write":826,"num_write_kb":7851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":139311,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":203076,"kb_used_data":3464,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566027964,"statfs":{"total":579820584960,"available":579612635136,"internally_reserved":0,"allocated":3547136,"data_stored":5278763,"data_compressed":921761,"data_compressed_allocated":1241088,"data_compressed_original":4411392,"omap_allocated":9568,"internal_metadata":204266144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":76800,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":486,"num_read_kb":420,"num_write":7,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.339206"},"pg_stats":[{"pgid":"3.1f","version":"24'1","reported_seq":45,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.874197+0000","last_change":"2024-09-14T07:07:49.800921+0000","last_active":"2024-09-14T07:07:59.874197+0000","last_peered":"2024-09-14T07:07:59.874197+0000","last_clean":"2024-09-14T07:07:59.874197+0000","last_became_active":"2024-09-14T07:07:47.794275+0000","last_became_peered":"2024-09-14T07:07:47.794275+0000","last_unstale":"2024-09-14T07:07:59.874197+0000","last_undegraded":"2024-09-14T07:07:59.874197+0000","last_fullsized":"2024-09-14T07:07:59.874197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:00:15.758814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":4,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"31'45","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.156746+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:08:00.156746+0000","last_peered":"2024-09-14T07:08:00.156746+0000","last_clean":"2024-09-14T07:08:00.156746+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:08:00.156746+0000","last_undegraded":"2024-09-14T07:08:00.156746+0000","last_fullsized":"2024-09-14T07:08:00.156746+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:24:07.842771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":5972116,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":217,"num_read_kb":883,"num_write":58,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'125","reported_seq":1497,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.910798+0000","last_change":"2024-09-14T07:07:49.819484+0000","last_active":"2024-09-14T07:07:59.910798+0000","last_peered":"2024-09-14T07:07:59.910798+0000","last_clean":"2024-09-14T07:07:59.910798+0000","last_became_active":"2024-09-14T07:07:47.787620+0000","last_became_peered":"2024-09-14T07:07:47.787620+0000","last_unstale":"2024-09-14T07:07:59.910798+0000","last_undegraded":"2024-09-14T07:07:59.910798+0000","last_fullsized":"2024-09-14T07:07:59.910798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:31:18.042481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":4808704,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2666,"num_read_kb":2498,"num_write":57,"num_write_kb":196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"29'5","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.910559+0000","last_change":"2024-09-14T07:07:49.820020+0000","last_active":"2024-09-14T07:07:59.910559+0000","last_peered":"2024-09-14T07:07:59.910559+0000","last_clean":"2024-09-14T07:07:59.910559+0000","last_became_active":"2024-09-14T07:07:47.797935+0000","last_became_peered":"2024-09-14T07:07:47.797935+0000","last_unstale":"2024-09-14T07:07:59.910559+0000","last_undegraded":"2024-09-14T07:07:59.910559+0000","last_fullsized":"2024-09-14T07:07:59.910559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:58.657528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"28'44","reported_seq":879,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.191012+0000","last_change":"2024-09-14T07:07:49.801349+0000","last_active":"2024-09-14T07:08:00.191012+0000","last_peered":"2024-09-14T07:08:00.191012+0000","last_clean":"2024-09-14T07:08:00.191012+0000","last_became_active":"2024-09-14T07:07:47.789185+0000","last_became_peered":"2024-09-14T07:07:47.789185+0000","last_unstale":"2024-09-14T07:08:00.191012+0000","last_undegraded":"2024-09-14T07:08:00.191012+0000","last_fullsized":"2024-09-14T07:08:00.191012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:23.703965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":19173888,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":802,"num_read_kb":2836,"num_write":22,"num_write_kb":491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"32'84","reported_seq":1284,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T07:08:02.032416+0000","last_change":"2024-09-14T07:07:49.810147+0000","last_active":"2024-09-14T07:08:02.032416+0000","last_peered":"2024-09-14T07:08:02.032416+0000","last_clean":"2024-09-14T07:08:02.032416+0000","last_became_active":"2024-09-14T07:07:47.794385+0000","last_became_peered":"2024-09-14T07:07:47.794385+0000","last_unstale":"2024-09-14T07:08:02.032416+0000","last_undegraded":"2024-09-14T07:08:02.032416+0000","last_fullsized":"2024-09-14T07:08:02.032416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:40:34.576744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2356,"num_read_kb":2063,"num_write":44,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"29'49","reported_seq":166,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868440+0000","last_change":"2024-09-14T07:07:49.812943+0000","last_active":"2024-09-14T07:07:59.868440+0000","last_peered":"2024-09-14T07:07:59.868440+0000","last_clean":"2024-09-14T07:07:59.868440+0000","last_became_active":"2024-09-14T07:07:47.792709+0000","last_became_peered":"2024-09-14T07:07:47.792709+0000","last_unstale":"2024-09-14T07:07:59.868440+0000","last_undegraded":"2024-09-14T07:07:59.868440+0000","last_fullsized":"2024-09-14T07:07:59.868440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:10:06.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":2576422,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":265,"num_read_kb":342,"num_write":64,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"26'5","reported_seq":72,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.877510+0000","last_change":"2024-09-14T07:07:53.859978+0000","last_active":"2024-09-14T07:07:59.877510+0000","last_peered":"2024-09-14T07:07:59.877510+0000","last_clean":"2024-09-14T07:07:59.877510+0000","last_became_active":"2024-09-14T07:07:53.840766+0000","last_became_peered":"2024-09-14T07:07:53.840766+0000","last_unstale":"2024-09-14T07:07:59.877510+0000","last_undegraded":"2024-09-14T07:07:59.877510+0000","last_fullsized":"2024-09-14T07:07:59.877510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:58.377084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":8192,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":16,"num_write":5,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"24'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.874240+0000","last_change":"2024-09-14T07:07:49.801370+0000","last_active":"2024-09-14T07:07:59.874240+0000","last_peered":"2024-09-14T07:07:59.874240+0000","last_clean":"2024-09-14T07:07:59.874240+0000","last_became_active":"2024-09-14T07:07:47.780206+0000","last_became_peered":"2024-09-14T07:07:47.780206+0000","last_unstale":"2024-09-14T07:07:59.874240+0000","last_undegraded":"2024-09-14T07:07:59.874240+0000","last_fullsized":"2024-09-14T07:07:59.874240+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:30:16.549462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.910953+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:07:59.910953+0000","last_peered":"2024-09-14T07:07:59.910953+0000","last_clean":"2024-09-14T07:07:59.910953+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:07:59.910953+0000","last_undegraded":"2024-09-14T07:07:59.910953+0000","last_fullsized":"2024-09-14T07:07:59.910953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:48.037727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"31'25","reported_seq":392,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.178767+0000","last_change":"2024-09-14T07:07:49.804465+0000","last_active":"2024-09-14T07:08:00.178767+0000","last_peered":"2024-09-14T07:08:00.178767+0000","last_clean":"2024-09-14T07:08:00.178767+0000","last_became_active":"2024-09-14T07:07:47.804440+0000","last_became_peered":"2024-09-14T07:07:47.804440+0000","last_unstale":"2024-09-14T07:08:00.178767+0000","last_undegraded":"2024-09-14T07:08:00.178767+0000","last_fullsized":"2024-09-14T07:08:00.178767+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:33.108246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":3223601,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":388,"num_read_kb":469,"num_write":17,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"31'26","reported_seq":178,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.997976+0000","last_change":"2024-09-14T07:07:49.803381+0000","last_active":"2024-09-14T07:07:59.997976+0000","last_peered":"2024-09-14T07:07:59.997976+0000","last_clean":"2024-09-14T07:07:59.997976+0000","last_became_active":"2024-09-14T07:07:47.794784+0000","last_became_peered":"2024-09-14T07:07:47.794784+0000","last_unstale":"2024-09-14T07:07:59.997976+0000","last_undegraded":"2024-09-14T07:07:59.997976+0000","last_fullsized":"2024-09-14T07:07:59.997976+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:10:24.283800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":11814400,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":176,"num_write":12,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868424+0000","last_change":"2024-09-14T07:07:38.031294+0000","last_active":"2024-09-14T07:07:59.868424+0000","last_peered":"2024-09-14T07:07:59.868424+0000","last_clean":"2024-09-14T07:07:59.868424+0000","last_became_active":"2024-09-14T07:07:36.418930+0000","last_became_peered":"2024-09-14T07:07:36.418930+0000","last_unstale":"2024-09-14T07:07:59.868424+0000","last_undegraded":"2024-09-14T07:07:59.868424+0000","last_fullsized":"2024-09-14T07:07:59.868424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:08.172090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"26'4","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.874173+0000","last_change":"2024-09-14T07:07:49.801738+0000","last_active":"2024-09-14T07:07:59.874173+0000","last_peered":"2024-09-14T07:07:59.874173+0000","last_clean":"2024-09-14T07:07:59.874173+0000","last_became_active":"2024-09-14T07:07:47.793157+0000","last_became_peered":"2024-09-14T07:07:47.793157+0000","last_unstale":"2024-09-14T07:07:59.874173+0000","last_undegraded":"2024-09-14T07:07:59.874173+0000","last_fullsized":"2024-09-14T07:07:59.874173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:30.313038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":36,"num_write":4,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868748+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:07:59.868748+0000","last_peered":"2024-09-14T07:07:59.868748+0000","last_clean":"2024-09-14T07:07:59.868748+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:07:59.868748+0000","last_undegraded":"2024-09-14T07:07:59.868748+0000","last_fullsized":"2024-09-14T07:07:59.868748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"26'5","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.877562+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:07:59.877562+0000","last_peered":"2024-09-14T07:07:59.877562+0000","last_clean":"2024-09-14T07:07:59.877562+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:07:59.877562+0000","last_undegraded":"2024-09-14T07:07:59.877562+0000","last_fullsized":"2024-09-14T07:07:59.877562+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:11:00.466660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":243,"num_write":5,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868545+0000","last_change":"2024-09-14T07:07:38.031148+0000","last_active":"2024-09-14T07:07:59.868545+0000","last_peered":"2024-09-14T07:07:59.868545+0000","last_clean":"2024-09-14T07:07:59.868545+0000","last_became_active":"2024-09-14T07:07:36.196549+0000","last_became_peered":"2024-09-14T07:07:36.196549+0000","last_unstale":"2024-09-14T07:07:59.868545+0000","last_undegraded":"2024-09-14T07:07:59.868545+0000","last_fullsized":"2024-09-14T07:07:59.868545+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:35.911258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"31'6","reported_seq":56,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.130983+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:08:00.130983+0000","last_peered":"2024-09-14T07:08:00.130983+0000","last_clean":"2024-09-14T07:08:00.130983+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:08:00.130983+0000","last_undegraded":"2024-09-14T07:08:00.130983+0000","last_fullsized":"2024-09-14T07:08:00.130983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:10:45.638629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":8,"num_write":6,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868691+0000","last_change":"2024-09-14T07:07:38.032270+0000","last_active":"2024-09-14T07:07:59.868691+0000","last_peered":"2024-09-14T07:07:59.868691+0000","last_clean":"2024-09-14T07:07:59.868691+0000","last_became_active":"2024-09-14T07:07:36.196923+0000","last_became_peered":"2024-09-14T07:07:36.196923+0000","last_unstale":"2024-09-14T07:07:59.868691+0000","last_undegraded":"2024-09-14T07:07:59.868691+0000","last_fullsized":"2024-09-14T07:07:59.868691+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:20.173745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"31'12","reported_seq":604,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.186732+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:08:00.186732+0000","last_peered":"2024-09-14T07:08:00.186732+0000","last_clean":"2024-09-14T07:08:00.186732+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:08:00.186732+0000","last_undegraded":"2024-09-14T07:08:00.186732+0000","last_fullsized":"2024-09-14T07:08:00.186732+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:43:13.241020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":4954624,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":458,"num_read_kb":2824,"num_write":12,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"24'2","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T07:08:02.032517+0000","last_change":"2024-09-14T07:07:49.808620+0000","last_active":"2024-09-14T07:08:02.032517+0000","last_peered":"2024-09-14T07:08:02.032517+0000","last_clean":"2024-09-14T07:08:02.032517+0000","last_became_active":"2024-09-14T07:07:47.786098+0000","last_became_peered":"2024-09-14T07:07:47.786098+0000","last_unstale":"2024-09-14T07:08:02.032517+0000","last_undegraded":"2024-09-14T07:08:02.032517+0000","last_fullsized":"2024-09-14T07:08:02.032517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:32:00.243099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.910466+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:07:59.910466+0000","last_peered":"2024-09-14T07:07:59.910466+0000","last_clean":"2024-09-14T07:07:59.910466+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:07:59.910466+0000","last_undegraded":"2024-09-14T07:07:59.910466+0000","last_fullsized":"2024-09-14T07:07:59.910466+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"29'1","reported_seq":49,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868579+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:07:59.868579+0000","last_peered":"2024-09-14T07:07:59.868579+0000","last_clean":"2024-09-14T07:07:59.868579+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:07:59.868579+0000","last_undegraded":"2024-09-14T07:07:59.868579+0000","last_fullsized":"2024-09-14T07:07:59.868579+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:46.114792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":3,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.877486+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:07:59.877486+0000","last_peered":"2024-09-14T07:07:59.877486+0000","last_clean":"2024-09-14T07:07:59.877486+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:07:59.877486+0000","last_undegraded":"2024-09-14T07:07:59.877486+0000","last_fullsized":"2024-09-14T07:07:59.877486+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"31'5","reported_seq":87,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.874019+0000","last_change":"2024-09-14T07:07:49.801096+0000","last_active":"2024-09-14T07:07:59.874019+0000","last_peered":"2024-09-14T07:07:59.874019+0000","last_clean":"2024-09-14T07:07:59.874019+0000","last_became_active":"2024-09-14T07:07:47.789827+0000","last_became_peered":"2024-09-14T07:07:47.789827+0000","last_unstale":"2024-09-14T07:07:59.874019+0000","last_undegraded":"2024-09-14T07:07:59.874019+0000","last_fullsized":"2024-09-14T07:07:59.874019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:40.653194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":244,"num_write":4,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.867772+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:07:59.867772+0000","last_peered":"2024-09-14T07:07:59.867772+0000","last_clean":"2024-09-14T07:07:59.867772+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:07:59.867772+0000","last_undegraded":"2024-09-14T07:07:59.867772+0000","last_fullsized":"2024-09-14T07:07:59.867772+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"31'29","reported_seq":234,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.130133+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:08:00.130133+0000","last_peered":"2024-09-14T07:08:00.130133+0000","last_clean":"2024-09-14T07:08:00.130133+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:08:00.130133+0000","last_undegraded":"2024-09-14T07:08:00.130133+0000","last_fullsized":"2024-09-14T07:08:00.130133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:50:59.072861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":524304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":177,"num_read_kb":273,"num_write":24,"num_write_kb":361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":65,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.910707+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:07:59.910707+0000","last_peered":"2024-09-14T07:07:59.910707+0000","last_clean":"2024-09-14T07:07:59.910707+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:07:59.910707+0000","last_undegraded":"2024-09-14T07:07:59.910707+0000","last_fullsized":"2024-09-14T07:07:59.910707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":135,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.867791+0000","last_change":"2024-09-14T07:07:32.604869+0000","last_active":"2024-09-14T07:07:59.867791+0000","last_peered":"2024-09-14T07:07:59.867791+0000","last_clean":"2024-09-14T07:07:59.867791+0000","last_became_active":"2024-09-14T07:07:32.602241+0000","last_became_peered":"2024-09-14T07:07:32.602241+0000","last_unstale":"2024-09-14T07:07:59.867791+0000","last_undegraded":"2024-09-14T07:07:59.867791+0000","last_fullsized":"2024-09-14T07:07:59.867791+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:25.905190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"31'21","reported_seq":155,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.137485+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:08:00.137485+0000","last_peered":"2024-09-14T07:08:00.137485+0000","last_clean":"2024-09-14T07:08:00.137485+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:08:00.137485+0000","last_undegraded":"2024-09-14T07:08:00.137485+0000","last_fullsized":"2024-09-14T07:08:00.137485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:25:43.063701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":7600640,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":387,"num_write":17,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"31'143","reported_seq":196,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-14T07:08:02.032472+0000","last_change":"2024-09-14T07:07:53.858015+0000","last_active":"2024-09-14T07:08:02.032472+0000","last_peered":"2024-09-14T07:08:02.032472+0000","last_clean":"2024-09-14T07:08:02.032472+0000","last_became_active":"2024-09-14T07:07:53.839838+0000","last_became_peered":"2024-09-14T07:07:53.839838+0000","last_unstale":"2024-09-14T07:08:02.032472+0000","last_undegraded":"2024-09-14T07:08:02.032472+0000","last_fullsized":"2024-09-14T07:08:02.032472+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:55.628216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":76849,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":688,"num_read_kb":549,"num_write":235,"num_write_kb":342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":47,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"29'16","reported_seq":133,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868034+0000","last_change":"2024-09-14T07:07:49.803037+0000","last_active":"2024-09-14T07:07:59.868034+0000","last_peered":"2024-09-14T07:07:59.868034+0000","last_clean":"2024-09-14T07:07:59.868034+0000","last_became_active":"2024-09-14T07:07:47.801723+0000","last_became_peered":"2024-09-14T07:07:47.801723+0000","last_unstale":"2024-09-14T07:07:59.868034+0000","last_undegraded":"2024-09-14T07:07:59.868034+0000","last_fullsized":"2024-09-14T07:07:59.868034+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:41:08.761600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":1452032,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":139,"num_write":16,"num_write_kb":295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"31'3","reported_seq":43,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.868736+0000","last_change":"2024-09-14T07:07:49.813689+0000","last_active":"2024-09-14T07:07:59.868736+0000","last_peered":"2024-09-14T07:07:59.868736+0000","last_clean":"2024-09-14T07:07:59.868736+0000","last_became_active":"2024-09-14T07:07:47.803551+0000","last_became_peered":"2024-09-14T07:07:47.803551+0000","last_unstale":"2024-09-14T07:07:59.868736+0000","last_undegraded":"2024-09-14T07:07:59.868736+0000","last_fullsized":"2024-09-14T07:07:59.868736+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:23.852623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":38,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":3,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"24'2","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.911147+0000","last_change":"2024-09-14T07:07:49.820702+0000","last_active":"2024-09-14T07:07:59.911147+0000","last_peered":"2024-09-14T07:07:59.911147+0000","last_clean":"2024-09-14T07:07:59.911147+0000","last_became_active":"2024-09-14T07:07:47.781704+0000","last_became_peered":"2024-09-14T07:07:47.781704+0000","last_unstale":"2024-09-14T07:07:59.911147+0000","last_undegraded":"2024-09-14T07:07:59.911147+0000","last_fullsized":"2024-09-14T07:07:59.911147+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:45:22.957079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"31'32","reported_seq":133,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.056801+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:08:00.056801+0000","last_peered":"2024-09-14T07:08:00.056801+0000","last_clean":"2024-09-14T07:08:00.056801+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:08:00.056801+0000","last_undegraded":"2024-09-14T07:08:00.056801+0000","last_fullsized":"2024-09-14T07:08:00.056801+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:12.431344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":1048576,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":24,"num_write":22,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"24'3","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.911152+0000","last_change":"2024-09-14T07:07:49.820959+0000","last_active":"2024-09-14T07:07:59.911152+0000","last_peered":"2024-09-14T07:07:59.911152+0000","last_clean":"2024-09-14T07:07:59.911152+0000","last_became_active":"2024-09-14T07:07:47.786208+0000","last_became_peered":"2024-09-14T07:07:47.786208+0000","last_unstale":"2024-09-14T07:07:59.911152+0000","last_undegraded":"2024-09-14T07:07:59.911152+0000","last_fullsized":"2024-09-14T07:07:59.911152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:29:12.003731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"31'1","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.134354+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:08:00.134354+0000","last_peered":"2024-09-14T07:08:00.134354+0000","last_clean":"2024-09-14T07:08:00.134354+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:08:00.134354+0000","last_undegraded":"2024-09-14T07:08:00.134354+0000","last_fullsized":"2024-09-14T07:08:00.134354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:12.028296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":16,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"28'18","reported_seq":132,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.995493+0000","last_change":"2024-09-14T07:07:49.818154+0000","last_active":"2024-09-14T07:07:59.995493+0000","last_peered":"2024-09-14T07:07:59.995493+0000","last_clean":"2024-09-14T07:07:59.995493+0000","last_became_active":"2024-09-14T07:07:47.782423+0000","last_became_peered":"2024-09-14T07:07:47.782423+0000","last_unstale":"2024-09-14T07:07:59.995493+0000","last_undegraded":"2024-09-14T07:07:59.995493+0000","last_fullsized":"2024-09-14T07:07:59.995493+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:15:29.150116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":7864320,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":200,"num_write":19,"num_write_kb":400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"28'20","reported_seq":167,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:08:00.087243+0000","last_change":"2024-09-14T07:07:49.802393+0000","last_active":"2024-09-14T07:08:00.087243+0000","last_peered":"2024-09-14T07:08:00.087243+0000","last_clean":"2024-09-14T07:08:00.087243+0000","last_became_active":"2024-09-14T07:07:47.791132+0000","last_became_peered":"2024-09-14T07:07:47.791132+0000","last_unstale":"2024-09-14T07:08:00.087243+0000","last_undegraded":"2024-09-14T07:08:00.087243+0000","last_fullsized":"2024-09-14T07:08:00.087243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:28:19.858807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":17920512,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":312,"num_write":15,"num_write_kb":310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"28'27","reported_seq":299,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.874000+0000","last_change":"2024-09-14T07:07:49.800610+0000","last_active":"2024-09-14T07:07:59.874000+0000","last_peered":"2024-09-14T07:07:59.874000+0000","last_clean":"2024-09-14T07:07:59.874000+0000","last_became_active":"2024-09-14T07:07:47.795686+0000","last_became_peered":"2024-09-14T07:07:47.795686+0000","last_unstale":"2024-09-14T07:07:59.874000+0000","last_undegraded":"2024-09-14T07:07:59.874000+0000","last_fullsized":"2024-09-14T07:07:59.874000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:15.966848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":18746368,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":237,"num_read_kb":482,"num_write":25,"num_write_kb":440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"28'25","reported_seq":120,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-14T07:07:59.867978+0000","last_change":"2024-09-14T07:07:49.804715+0000","last_active":"2024-09-14T07:07:59.867978+0000","last_peered":"2024-09-14T07:07:59.867978+0000","last_clean":"2024-09-14T07:07:59.867978+0000","last_became_active":"2024-09-14T07:07:47.787058+0000","last_became_peered":"2024-09-14T07:07:47.787058+0000","last_unstale":"2024-09-14T07:07:59.867978+0000","last_undegraded":"2024-09-14T07:07:59.867978+0000","last_fullsized":"2024-09-14T07:07:59.867978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:33:15.706191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":9146368,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":61,"num_read_kb":405,"num_write":20,"num_write_kb":515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":121368453,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":9081,"num_read_kb":15450,"num_write":723,"num_write_kb":6221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":139311,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1527808,"data_stored":3533726,"data_compressed":909876,"data_compressed_allocated":1179648,"data_compressed_original":3399680,"omap_allocated":0,"internal_metadata":0},"log_size":786,"ondisk_log_size":786,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35908,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335932,"statfs":{"total":96636764160,"available":96599994368,"internally_reserved":0,"allocated":573440,"data_stored":745333,"data_compressed":134844,"data_compressed_allocated":180224,"data_compressed_original":598016,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607562,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35812,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336028,"statfs":{"total":96636764160,"available":96600092672,"internally_reserved":0,"allocated":475136,"data_stored":467677,"data_compressed":86705,"data_compressed_allocated":114688,"data_compressed_original":331776,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607561,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36132,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335708,"statfs":{"total":96636764160,"available":96599764992,"internally_reserved":0,"allocated":798720,"data_stored":1602913,"data_compressed":273623,"data_compressed_allocated":376832,"data_compressed_original":1433600,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607563,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":602112,"data_stored":826347,"data_compressed":159177,"data_compressed_allocated":212992,"data_compressed_original":684032,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607563,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":440652,"data_compressed":112714,"data_compressed_allocated":135168,"data_compressed_original":315392,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607563,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":602112,"data_stored":1195841,"data_compressed":154698,"data_compressed_allocated":221184,"data_compressed_original":1048576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":489575,"data_compressed":148640,"data_compressed_allocated":184320,"data_compressed_original":475136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":324735,"data_compressed":112714,"data_compressed_allocated":135168,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":815829,"data_compressed":159339,"data_compressed_allocated":221184,"data_compressed_original":782336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":891605,"data_compressed":267565,"data_compressed_allocated":339968,"data_compressed_original":860160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":619262,"data_compressed":134844,"data_compressed_allocated":180224,"data_compressed_original":598016,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":392720,"data_compressed":86774,"data_compressed_allocated":118784,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:08:05.697 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [1, 4, 3, 5] 2024-09-14T07:08:05.697 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.3', '1', '4', '3', '5'] 2024-09-14T07:08:05.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.3 1 4 3 5 2024-09-14T07:08:05.705 INFO:teuthology.orchestra.run.smithi145.stdout:494 trunc from 0x71af600 to 0x431fa00 2024-09-14T07:08:05.741 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:05.738+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:05.796 INFO:teuthology.orchestra.run.smithi145.stdout:495 write 0xb343600 thru 0xb351b58 (0xe559 bytes) 2024-09-14T07:08:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:496 write 0x4af4a00 thru 0x4af7c37 (0x3238 bytes) 2024-09-14T07:08:05.798 INFO:teuthology.orchestra.run.smithi145.stdout:497 read 0x980c000 thru 0x981a195 (0xe196 bytes) 2024-09-14T07:08:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:498 write 0x59f1000 thru 0x59fec30 (0xdc31 bytes) 2024-09-14T07:08:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:501 read 0x1061600 thru 0x106d141 (0xbb42 bytes) 2024-09-14T07:08:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:503 write 0x823600 thru 0x82587f (0x2280 bytes) 2024-09-14T07:08:05.799 INFO:teuthology.orchestra.run.smithi145.stdout:505 write 0x9a5f200 thru 0x9a627f4 (0x35f5 bytes) 2024-09-14T07:08:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:506 write 0xa938200 thru 0xa93d51f (0x5320 bytes) 2024-09-14T07:08:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:507 read 0xc21a00 thru 0xc2882a (0x6e2b bytes) 2024-09-14T07:08:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:512 read 0x8daa200 thru 0x8db75fe (0xd3ff bytes) 2024-09-14T07:08:05.800 INFO:teuthology.orchestra.run.smithi145.stdout:513 write 0x5b70a00 thru 0x5b7f71a (0xed1b bytes) 2024-09-14T07:08:05.808 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:05.809 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:05.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:05.842 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:05.838+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:05.864 INFO:teuthology.orchestra.run.smithi145.stdout:514 write 0xe530e00 thru 0xe53f1c3 (0xe3c4 bytes) 2024-09-14T07:08:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:515 write 0x9ef7800 thru 0x9ef9346 (0x1b47 bytes) 2024-09-14T07:08:05.868 INFO:teuthology.orchestra.run.smithi145.stdout:517 write 0x5d10800 thru 0x5d14956 (0x4157 bytes) 2024-09-14T07:08:05.884 INFO:teuthology.orchestra.run.smithi145.stdout:518 trunc from 0xe53f200 to 0x717cc00 2024-09-14T07:08:05.928 INFO:teuthology.orchestra.run.smithi145.stdout:521 clone 5 order 19 su 8192 sc 12 2024-09-14T07:08:05.943 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:05.942+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:05.971 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:05.972 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:05.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:06.044 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:06.042+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:06.113 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:06.113 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:06.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:06.145 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:06.142+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:06.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:06.247 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:06.246+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:06.318 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:06.347 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:06.346+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:06.386 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:06.403 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:06.404 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:06.420 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:06.441 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:06.447 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:06.448 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:06.448 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:06.448 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:06.448 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:06.448 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:06.446+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:06.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:06.463 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:06.483 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:06.549 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:06.546+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:06.583 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:06.583 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:06.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:06.583 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:06.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.583 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:832 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.583 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106514+0000", 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4751967399999999, 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047025360000000002, 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 832 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106514+0000", 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106514+0000", 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106517+0000", 2024-09-14T07:08:06.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2619999999999999e-06 2024-09-14T07:08:06.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:06.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:08:06.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106533+0000", 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1824999999999999e-05 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106541+0000", 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5279999999999998e-06 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153376+0000", 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046835379000000003 2024-09-14T07:08:06.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153450+0000", 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3460999999999993e-05 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153539+0000", 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9641000000000005e-05 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1611 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.475188357, 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047250024000000002, 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1611 2024-09-14T07:08:06.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106523+0000", 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106535+0000", 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1606e-05 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106544+0000", 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7369999999999995e-06 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106548+0000", 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5379999999999996e-06 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153573+0000", 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047024375 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153654+0000", 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1230000000000007e-05 2024-09-14T07:08:06.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153772+0000", 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118558 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1612 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4750280980000001, 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047314770999999999, 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1612 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9599999999999998e-07 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106687+0000", 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7759999999999999e-06 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106696+0000", 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0129999999999999e-06 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4431e-05 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153824+0000", 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047113006999999998 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153901+0000", 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7015999999999996e-05 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153997+0000", 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6731999999999995e-05 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:833 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4749996300000001, 2024-09-14T07:08:06.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047546469000000001, 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 833 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106712+0000", 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.043e-06 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106716+0000", 2024-09-14T07:08:06.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3579999999999996e-06 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106728+0000", 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1534e-05 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106735+0000", 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2740000000000002e-06 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154044+0000", 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047308463000000002 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154141+0000", 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6903000000000004e-05 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154258+0000", 2024-09-14T07:08:06.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116894 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:834 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4748791620000001, 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047638265999999999, 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 834 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:06.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5700000000000002e-07 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106836+0000", 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0489999999999997e-06 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106854+0000", 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7632e-05 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106862+0000", 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154302+0000", 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047440309 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154375+0000", 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2943999999999998e-05 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154470+0000", 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4829999999999998e-05 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1613 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106902+0000", 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.474808559, 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047815349, 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1613 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.607 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106902+0000", 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106902+0000", 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106903+0000", 2024-09-14T07:08:06.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5600000000000005e-07 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106906+0000", 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106914+0000", 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0990000000000001e-06 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106918+0000", 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5050000000000001e-06 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154517+0000", 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047598616000000003 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154600+0000", 2024-09-14T07:08:06.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2868999999999996e-05 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154718+0000", 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117973 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1614 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107048+0000", 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4746628479999999, 2024-09-14T07:08:06.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047890478, 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1614 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107048+0000", 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107048+0000", 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107049+0000", 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4200000000000005e-07 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107053+0000", 2024-09-14T07:08:06.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2309999999999998e-06 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107063+0000", 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9049999999999992e-06 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107070+0000", 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6610000000000003e-06 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154768+0000", 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047698590999999999 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154840+0000", 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2015999999999996e-05 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154938+0000", 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8232000000000004e-05 2024-09-14T07:08:06.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:835 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107120+0000", 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4745907950000001, 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048072937000000003, 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 835 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107120+0000", 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107120+0000", 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107121+0000", 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.015e-06 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107125+0000", 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4800000000000003e-06 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107137+0000", 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1542000000000001e-05 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107144+0000", 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3919999999999997e-06 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154985+0000", 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047840420000000002 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155080+0000", 2024-09-14T07:08:06.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4868e-05 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155193+0000", 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011322 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1615 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107193+0000", 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4745177300000001, 2024-09-14T07:08:06.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048212334000000003, 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1615 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107193+0000", 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107193+0000", 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107194+0000", 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1200000000000003e-07 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107197+0000", 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4249999999999998e-06 2024-09-14T07:08:06.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107204+0000", 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3930000000000002e-06 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107210+0000", 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5500000000000002e-06 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155237+0000", 2024-09-14T07:08:06.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048026712999999999 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155314+0000", 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6936999999999999e-05 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155405+0000", 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1704000000000002e-05 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:836 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107211+0000", 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4744993109999998, 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048451056999999999, 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 836 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.625 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107211+0000", 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107211+0000", 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107212+0000", 2024-09-14T07:08:06.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.037e-06 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107217+0000", 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107227+0000", 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.024e-05 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107234+0000", 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1239999999999999e-06 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155456+0000", 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048221676999999998 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155539+0000", 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2878000000000002e-05 2024-09-14T07:08:06.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155662+0000", 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000123681 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:837 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4742691309999998, 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048434352, 2024-09-14T07:08:06.629 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 837 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2900000000000003e-07 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107447+0000", 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107462+0000", 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4948e-05 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107469+0000", 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6830000000000008e-06 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155707+0000", 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048237774999999997 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155783+0000", 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155876+0000", 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3113000000000001e-05 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.634 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1616 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:06.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.474156153, 2024-09-14T07:08:06.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048566559000000002, 2024-09-14T07:08:06.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1616 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:06.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8100000000000002e-07 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107559+0000", 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3799999999999998e-06 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107567+0000", 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.994e-06 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107571+0000", 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6959999999999996e-06 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155923+0000", 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048351466000000003 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156005+0000", 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2137000000000003e-05 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156121+0000", 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116205 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1617 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107603+0000", 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.474107241, 2024-09-14T07:08:06.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048741325000000002, 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1617 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107603+0000", 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107603+0000", 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107604+0000", 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8800000000000002e-07 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107608+0000", 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4860000000000002e-06 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107616+0000", 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8040000000000007e-06 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107631+0000", 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4511999999999999e-05 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156181+0000", 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048549595000000001 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156252+0000", 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1250999999999999e-05 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156345+0000", 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2989e-05 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.644 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:838 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107605+0000", 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4741054359999999, 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048990005000000003, 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 838 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107605+0000", 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107605+0000", 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107606+0000", 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107611+0000", 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107632+0000", 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1202000000000001e-05 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107641+0000", 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1750000000000005e-06 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156394+0000", 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048753071000000002 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156477+0000", 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3543999999999998e-05 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156595+0000", 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118053 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:839 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107741+0000", 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4739701329999999, 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049065575, 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 839 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.649 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107741+0000", 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107741+0000", 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107742+0000", 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9099999999999991e-07 2024-09-14T07:08:06.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107747+0000", 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9159999999999998e-06 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107759+0000", 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1544e-05 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107770+0000", 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:08:06.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156640+0000", 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048869888 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156714+0000", 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3981999999999997e-05 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156806+0000", 2024-09-14T07:08:06.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2567999999999999e-05 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1618 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107758+0000", 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4739525260000002, 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049297046999999997, 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.653 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1618 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107758+0000", 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107758+0000", 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107759+0000", 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.251e-06 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107764+0000", 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.746e-06 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107773+0000", 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1409999999999994e-06 2024-09-14T07:08:06.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107780+0000", 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4529999999999999e-06 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156852+0000", 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049072431 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156934+0000", 2024-09-14T07:08:06.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1589999999999996e-05 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157055+0000", 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012143500000000001 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1619 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.657 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.473737211, 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049542967, 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1619 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.61e-07 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107986+0000", 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2136e-05 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107995+0000", 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9779999999999994e-06 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108002+0000", 2024-09-14T07:08:06.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0149999999999998e-06 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157318+0000", 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049315921999999998 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157399+0000", 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0793999999999999e-05 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157516+0000", 2024-09-14T07:08:06.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117361 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:840 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4737366110000001, 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049296567999999999, 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 840 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107975+0000", 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.246e-06 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107980+0000", 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7809999999999996e-06 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107992+0000", 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1415e-05 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107999+0000", 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4259999999999997e-06 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157108+0000", 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04910867 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157181+0000", 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2887999999999997e-05 2024-09-14T07:08:06.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157271+0000", 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0142000000000005e-05 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:841 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.473450186, 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049470640000000003, 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 841 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999998e-07 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108269+0000", 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108282+0000", 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2318e-05 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108289+0000", 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6729999999999992e-06 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157564+0000", 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049274577999999999 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157641+0000", 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6814e-05 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157731+0000", 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1620 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.108352+0000", 2024-09-14T07:08:06.670 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.4733591229999998, 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049620367999999998, 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1620 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108352+0000", 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108352+0000", 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108353+0000", 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1900000000000001e-07 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108357+0000", 2024-09-14T07:08:06.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4240000000000003e-06 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108369+0000", 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2269999999999999e-05 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108376+0000", 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157776+0000", 2024-09-14T07:08:06.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049399817999999998 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157859+0000", 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2640999999999997e-05 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157972+0000", 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113074 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:06.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:06.675 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:06.676 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:06.677 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:06.674+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:06.777 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:06.774+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:06.807 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:06.807 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:06.821 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:06.878 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:06.878+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.3 pg_upmap_items mapping to [1->4,3->5] 2024-09-14T07:08:06.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:08:06.979 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:06.978+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:06.998 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:06.999 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:06.999 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:06.999 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:06.999 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:07.014 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:07.048 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone4 from 0x717cc00 (overlap 0x431fa00) to 0x1ad9600 2024-09-14T07:08:07.099 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:07.098+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:07.181 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:07.178+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:07.195 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 34.893077611999999, 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:330 3.11 3:8d35ed92:::rbd_data.1161a9b4f57.000000000000005a:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.058709+0000", 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13487263699999999, 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028155896999999999, 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 330 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058709+0000", 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058709+0000", 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058710+0000", 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1500000000000004e-07 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058712+0000", 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.633e-06 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058721+0000", 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058728+0000", 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6040000000000004e-06 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.083979+0000", 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.025250742999999999 2024-09-14T07:08:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.084266+0000", 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000286839 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.086865+0000", 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025989770000000001 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:350 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify cookie 140557502733248] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.059133+0000", 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13444842500000001, 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027587212999999999, 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 350 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059133+0000", 2024-09-14T07:08:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059133+0000", 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059134+0000", 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1800000000000005e-07 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059138+0000", 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0910000000000003e-06 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059146+0000", 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3080000000000008e-06 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059152+0000", 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9599999999999997e-06 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.086470+0000", 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027317701999999999 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.086544+0000", 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3800000000000005e-05 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.086720+0000", 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000176634 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:351 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify cookie 140557502745120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.059162+0000", 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13441948300000001, 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027969253999999999, 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 351 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059162+0000", 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059162+0000", 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059163+0000", 2024-09-14T07:08:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2499999999999995e-07 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059166+0000", 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0280000000000001e-06 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059172+0000", 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7000000000000002e-06 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059178+0000", 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5550000000000002e-06 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.086926+0000", 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027748596 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087001+0000", 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4481999999999996e-05 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087131+0000", 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013036800000000001 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:352 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify cookie 140557502757632] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.059195+0000", 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.134386217, 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028182437000000001, 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 352 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059195+0000", 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059195+0000", 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059196+0000", 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059199+0000", 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6040000000000001e-06 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059208+0000", 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1680000000000004e-06 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059221+0000", 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3559e-05 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087181+0000", 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027959635 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087245+0000", 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4444999999999997e-05 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087378+0000", 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013240499999999999 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:1926 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.059209+0000", 2024-09-14T07:08:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13437259300000001, 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028370026999999999, 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1926 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059209+0000", 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059209+0000", 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059210+0000", 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-07 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059213+0000", 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7809999999999999e-06 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059222+0000", 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4319999999999999e-06 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059227+0000", 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087412+0000", 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028184563999999999 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087479+0000", 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7575000000000003e-05 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087579+0000", 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9691999999999994e-05 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:1927 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.059458+0000", 2024-09-14T07:08:07.223 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13412368099999999, 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028335394, 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1927 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059458+0000", 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059458+0000", 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059459+0000", 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059464+0000", 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3290000000000001e-06 2024-09-14T07:08:07.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059474+0000", 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8130000000000007e-06 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059481+0000", 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7739999999999999e-06 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087628+0000", 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028147597 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087701+0000", 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2299999999999996e-05 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087793+0000", 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2436999999999996e-05 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:1928 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.227 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.059983+0000", 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13359870600000001, 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028336120999999999, 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1928 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059983+0000", 2024-09-14T07:08:07.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059983+0000", 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059984+0000", 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0109999999999999e-06 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059988+0000", 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.194e-06 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.059999+0000", 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0706999999999999e-05 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060006+0000", 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2930000000000003e-06 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.087959+0000", 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027953186000000001 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088123+0000", 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016363400000000001 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088319+0000", 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019609599999999999 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.231 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:354 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.060743+0000", 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13283894299999999, 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027821788, 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.232 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 354 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060743+0000", 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060743+0000", 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060743+0000", 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060747+0000", 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0690000000000003e-06 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060756+0000", 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3080000000000008e-06 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060773+0000", 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7071e-05 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088386+0000", 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027612778000000001 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088457+0000", 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1860999999999995e-05 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088564+0000", 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010686799999999999 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:355 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.060787+0000", 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13279496199999999, 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028106339000000001, 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.236 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 355 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060787+0000", 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060787+0000", 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060787+0000", 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6599999999999996e-07 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060791+0000", 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3979999999999999e-06 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060798+0000", 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6979999999999998e-06 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060806+0000", 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9959999999999993e-06 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088602+0000", 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027796187 2024-09-14T07:08:07.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088775+0000", 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017248599999999999 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088893+0000", 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117908 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:356 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.060815+0000", 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13276606299999999, 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028288019000000001, 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 356 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060815+0000", 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060815+0000", 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060816+0000", 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5799999999999999e-07 2024-09-14T07:08:07.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060819+0000", 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8480000000000001e-06 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060825+0000", 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4829999999999996e-06 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060831+0000", 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9009999999999999e-06 2024-09-14T07:08:07.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088929+0000", 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028097989 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.088994+0000", 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4746999999999995e-05 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089103+0000", 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109493 2024-09-14T07:08:07.244 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:357 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.060839+0000", 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.132742376, 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028445729999999999, 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 357 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060839+0000", 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060839+0000", 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060840+0000", 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8699999999999995e-07 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060842+0000", 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3640000000000002e-06 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060849+0000", 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6640000000000001e-06 2024-09-14T07:08:07.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060854+0000", 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9849999999999997e-06 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089133+0000", 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028279426999999999 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089195+0000", 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1535e-05 2024-09-14T07:08:07.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089285+0000", 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0267999999999997e-05 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:1929 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.060903+0000", 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13267879599999999, 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028569889000000001, 2024-09-14T07:08:07.249 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.250 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.252 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.252 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1929 2024-09-14T07:08:07.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.252 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060903+0000", 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060903+0000", 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060903+0000", 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9599999999999999e-07 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060907+0000", 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999999e-06 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060916+0000", 2024-09-14T07:08:07.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9290000000000004e-06 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060934+0000", 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8224999999999999e-05 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089316+0000", 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028382333999999999 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089379+0000", 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2180000000000004e-05 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089473+0000", 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4019999999999998e-05 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:1930 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.060954+0000", 2024-09-14T07:08:07.256 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.132627203, 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028713873000000001, 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1930 2024-09-14T07:08:07.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060954+0000", 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060954+0000", 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060955+0000", 2024-09-14T07:08:07.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060957+0000", 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4059999999999999e-06 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060962+0000", 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3499999999999996e-06 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060966+0000", 2024-09-14T07:08:07.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.207e-06 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089509+0000", 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028543127000000001 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089594+0000", 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5457999999999994e-05 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089668+0000", 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3916000000000002e-05 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:360 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.261 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.060992+0000", 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13258993699999999, 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028989046000000001, 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 360 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060992+0000", 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060992+0000", 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060992+0000", 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.060994+0000", 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.131e-06 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061002+0000", 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5979999999999999e-06 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061005+0000", 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089730+0000", 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028725107999999999 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089836+0000", 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010538 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089981+0000", 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000144828 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.265 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:361 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.061011+0000", 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13257002500000001, 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029083500000000002, 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 361 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.266 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061011+0000", 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061011+0000", 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061012+0000", 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061013+0000", 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7010000000000001e-06 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061017+0000", 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000002e-06 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061020+0000", 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-14T07:08:07.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089997+0000", 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028977411000000002 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090039+0000", 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1730999999999997e-05 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090095+0000", 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5815e-05 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:362 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.061024+0000", 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13255772399999999, 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029399508000000001, 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061024+0000", 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061024+0000", 2024-09-14T07:08:07.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061024+0000", 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061041+0000", 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6993000000000001e-05 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061048+0000", 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-06 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061060+0000", 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1071e-05 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090141+0000", 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029081685999999999 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090283+0000", 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014164399999999999 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090423+0000", 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014037000000000001 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:366 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.063246+0000", 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.13033518499999999, 2024-09-14T07:08:07.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027743698000000001, 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 366 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063246+0000", 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063246+0000", 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063247+0000", 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063250+0000", 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063257+0000", 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063265+0000", 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-06 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090762+0000", 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027496791999999999 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090860+0000", 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8435000000000002e-05 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090990+0000", 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000129892 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:1931 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.063704+0000", 2024-09-14T07:08:07.278 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.12987754400000001, 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027611390999999999, 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1931 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063704+0000", 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063704+0000", 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063706+0000", 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063735+0000", 2024-09-14T07:08:07.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8819999999999999e-05 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063759+0000", 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3920000000000001e-05 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063778+0000", 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9446e-05 2024-09-14T07:08:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091037+0000", 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027258182999999998 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091161+0000", 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012443400000000001 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091315+0000", 2024-09-14T07:08:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000154365 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:1932 3.11 3:883b819f:::rbd_header.1161a9b4f57:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.063811+0000", 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.12977007500000001, 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027887821, 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1932 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063811+0000", 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063811+0000", 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063814+0000", 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1430000000000001e-06 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063822+0000", 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0739999999999993e-06 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063842+0000", 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9949000000000002e-05 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063856+0000", 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4436999999999999e-05 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091447+0000", 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027591460000000002 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091553+0000", 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010534100000000001 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091699+0000", 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000146417 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.287 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:07.289 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:07.289 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:07.286+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:07.389 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:07.386+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:07.469 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:07.470 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:07.470 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:07.470 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:07.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:07.531 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:07.530+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:07.564 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T07:08:07.590 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:07.590+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:07.607 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:07.607 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:07.607 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:07.607 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:07.607 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:07.621 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:07.711 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:07.710+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:07.791 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:07.790+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:07.804 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.009420507000002, 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070947240000000002, 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 7 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793553+0000", 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3730000000000001e-06 2024-09-14T07:08:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793583+0000", 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9675999999999998e-05 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793621+0000", 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7370000000000003e-05 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793647+0000", 2024-09-14T07:08:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6276999999999999e-05 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793835+0000", 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018812500000000001 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793852+0000", 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.734e-05 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797434+0000", 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035819189999999998 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797559+0000", 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012508500000000001 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797726+0000", 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000166797 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.798685+0000", 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00095928199999999999 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800546+0000", 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018609189999999999 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800602+0000", 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5365999999999998e-05 2024-09-14T07:08:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800645+0000", 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3195000000000002e-05 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.001986995999999, 2024-09-14T07:08:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0097102179999999996, 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800987+0000", 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801000+0000", 2024-09-14T07:08:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2897e-05 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801028+0000", 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8575999999999999e-05 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801054+0000", 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6367e-05 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801282+0000", 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022778100000000001 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801300+0000", 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7237000000000001e-05 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808053+0000", 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0067535249999999998 2024-09-14T07:08:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808316+0000", 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000263224 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808688+0000", 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037128699999999998 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810367+0000", 2024-09-14T07:08:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001679151 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810427+0000", 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0338999999999999e-05 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810572+0000", 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014462200000000001 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810694+0000", 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000122192 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:929 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.963400244000001, 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013302158999999999, 2024-09-14T07:08:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 929 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839572+0000", 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6780000000000001e-06 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839580+0000", 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0989999999999999e-06 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839590+0000", 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839598+0000", 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3699999999999997e-06 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840130+0000", 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00053217 2024-09-14T07:08:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840818+0000", 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00068814799999999997 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845641+0000", 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048226160000000001 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845730+0000", 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9687999999999999e-05 2024-09-14T07:08:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846136+0000", 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000405583 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849695+0000", 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00355887 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852725+0000", 2024-09-14T07:08:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030301009999999999 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852786+0000", 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1081999999999996e-05 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852873+0000", 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6608999999999997e-05 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:930 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.963360661999999, 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013501599, 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 930 2024-09-14T07:08:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839613+0000", 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4169999999999999e-06 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839618+0000", 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1499999999999998e-06 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839622+0000", 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.631e-06 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841189+0000", 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001567768 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841262+0000", 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2105000000000002e-05 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846495+0000", 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052335999999999997 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846559+0000", 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3609999999999996e-05 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846782+0000", 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022373299999999999 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849705+0000", 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002922998 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852940+0000", 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032343070000000001 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853014+0000", 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3967999999999994e-05 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853112+0000", 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7806000000000005e-05 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:931 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.963292357, 2024-09-14T07:08:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.01360341, 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 931 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:08:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839679+0000", 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7299999999999996e-07 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839682+0000", 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839688+0000", 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9569999999999999e-06 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839693+0000", 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6289999999999998e-06 2024-09-14T07:08:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841324+0000", 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016306090000000001 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841425+0000", 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010129299999999999 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847149+0000", 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057235139999999999 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847194+0000", 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5117000000000002e-05 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847441+0000", 2024-09-14T07:08:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000247428 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852669+0000", 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052276029999999999 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853162+0000", 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049292300000000004 2024-09-14T07:08:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853202+0000", 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0525999999999998e-05 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853282+0000", 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9513999999999998e-05 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:932 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.963185535000001, 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013718054, 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 932 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839786+0000", 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4499999999999996e-07 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839794+0000", 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6450000000000005e-06 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839805+0000", 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1489e-05 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839814+0000", 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6540000000000005e-06 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841579+0000", 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017656480000000001 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841669+0000", 2024-09-14T07:08:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9806e-05 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847963+0000", 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0062937080000000003 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848135+0000", 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017191299999999999 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848791+0000", 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00065584300000000003 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853128+0000", 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043374149999999998 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853351+0000", 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000223408 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853408+0000", 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6922000000000002e-05 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853503+0000", 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4758000000000003e-05 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:933 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.962754581, 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015741667000000001, 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 933 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:08:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840217+0000", 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1909999999999999e-06 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840230+0000", 2024-09-14T07:08:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3226e-05 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840244+0000", 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3108999999999999e-05 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840252+0000", 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2910000000000007e-06 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841706+0000", 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001454056 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841766+0000", 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0494000000000001e-05 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849824+0000", 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080573079999999991 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850028+0000", 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020432999999999999 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850748+0000", 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072041399999999995 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853135+0000", 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002386885 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855783+0000", 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026479350000000001 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855851+0000", 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7534999999999996e-05 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855958+0000", 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010689299999999999 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:935 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.961045443, 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016747201, 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 935 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841926+0000", 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841932+0000", 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2099999999999998e-06 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841943+0000", 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0981e-05 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841950+0000", 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842004+0000", 2024-09-14T07:08:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4422000000000002e-05 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842064+0000", 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9967999999999999e-05 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851737+0000", 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0096733600000000006 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851825+0000", 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7855999999999994e-05 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852258+0000", 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000432383 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853520+0000", 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001262787 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858562+0000", 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050418269999999996 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858623+0000", 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0789999999999999e-05 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858672+0000", 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9348999999999998e-05 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:936 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.960803685, 2024-09-14T07:08:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.018202472000000001, 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 936 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842168+0000", 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3300000000000002e-07 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842170+0000", 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6539999999999999e-06 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842178+0000", 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.413e-06 2024-09-14T07:08:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842182+0000", 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3139999999999997e-06 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842231+0000", 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8553000000000001e-05 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842290+0000", 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9534000000000003e-05 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853603+0000", 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011312579 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853796+0000", 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299000000000001 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854842+0000", 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00104612 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858690+0000", 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038478589999999999 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860251+0000", 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015616829999999999 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860315+0000", 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3587e-05 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860369+0000", 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4552999999999998e-05 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.866 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1142 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.785925487, 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.007319173, 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1142 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017046+0000", 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017061+0000", 2024-09-14T07:08:07.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.467e-05 2024-09-14T07:08:07.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017076+0000", 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5421e-05 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017088+0000", 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1287999999999999e-05 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019464+0000", 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023761590000000001 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019645+0000", 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000180772 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.020444+0000", 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00079905700000000004 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023077+0000", 2024-09-14T07:08:07.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002633093 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024178+0000", 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011013850000000001 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024296+0000", 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117586 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024364+0000", 2024-09-14T07:08:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8585999999999997e-05 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1145 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.785627166999999, 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077736239999999998, 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1145 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017346+0000", 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2369999999999999e-06 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017358+0000", 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2602999999999999e-05 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017390+0000", 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.188e-05 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017413+0000", 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.243e-05 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021308+0000", 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038951799999999998 2024-09-14T07:08:07.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021511+0000", 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020368099999999999 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.022262+0000", 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00075023799999999997 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023822+0000", 2024-09-14T07:08:07.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015606750000000001 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025018+0000", 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001195149 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8764999999999997e-05 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025117+0000", 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0785999999999999e-05 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1159 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.778001743000001, 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0074537930000000002, 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.882 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1159 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024970+0000", 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5939999999999999e-06 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024981+0000", 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:08:07.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025004+0000", 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3468999999999998e-05 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025021+0000", 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6674e-05 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025194+0000", 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173721 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025445+0000", 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025037099999999999 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.026330+0000", 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000885073 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032132+0000", 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058019819999999998 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032250+0000", 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118447 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032351+0000", 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100966 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032423+0000", 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1351000000000002e-05 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1160 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.77789581, 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0076908269999999999, 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.888 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1160 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.251e-06 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025084+0000", 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5689999999999999e-06 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025102+0000", 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8734000000000001e-05 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025116+0000", 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3465999999999999e-05 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027183+0000", 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002066877 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027355+0000", 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017235800000000001 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028074+0000", 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00071921500000000005 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032586+0000", 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045120960000000002 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032617+0000", 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0468999999999999e-05 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032697+0000", 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9939999999999997e-05 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032766+0000", 2024-09-14T07:08:07.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8851999999999997e-05 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1161 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.777797605, 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0078300520000000005, 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.894 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1161 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:08:07.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025174+0000", 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4139999999999999e-06 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025184+0000", 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3489999999999998e-06 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025201+0000", 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7011e-05 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025214+0000", 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3237e-05 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028886+0000", 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036715649999999999 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029039+0000", 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015298300000000001 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029611+0000", 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057197599999999997 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032782+0000", 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031712340000000002 2024-09-14T07:08:07.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032863+0000", 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0907000000000003e-05 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032957+0000", 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4715000000000004e-05 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033003+0000", 2024-09-14T07:08:07.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5661000000000002e-05 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1162 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.777729943000001, 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084900349999999999, 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1162 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:08:07.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025242+0000", 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3829999999999999e-06 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025248+0000", 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1999999999999999e-06 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025283+0000", 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4347000000000002e-05 2024-09-14T07:08:07.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025295+0000", 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2268e-05 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030146+0000", 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048513200000000001 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030309+0000", 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000163225 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.031281+0000", 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00097148600000000003 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033053+0000", 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001771941 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033582+0000", 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052929400000000003 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033653+0000", 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1285999999999995e-05 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033731+0000", 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7285000000000003e-05 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1167 3.a e26/21 3:509716c6:::rbd_data.11611b5ca054.000000000000002a:head v 26'12, mlcod=26'10)", 2024-09-14T07:08:07.905 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.768857353, 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065049039999999997, 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:08:07.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034114+0000", 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0800000000000003e-07 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034143+0000", 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8532000000000001e-05 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034158+0000", 2024-09-14T07:08:07.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5525000000000001e-05 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034169+0000", 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0726e-05 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034296+0000", 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012745700000000001 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034352+0000", 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5229999999999998e-05 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040532+0000", 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0061802130000000004 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040618+0000", 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6312999999999998e-05 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4539.0:2027 3.1a 3:59c1d47c:::rbd_header.116147128396:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:57.754109+0000", 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.048861297, 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006837419, 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4539", 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2491585557", 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2027 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.910 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754109+0000", 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754109+0000", 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754113+0000", 2024-09-14T07:08:07.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.32e-06 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754129+0000", 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6160000000000001e-05 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754169+0000", 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0583000000000001e-05 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754195+0000", 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5673e-05 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754283+0000", 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8084999999999994e-05 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754288+0000", 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6110000000000004e-06 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.757744+0000", 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034567270000000002 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.757988+0000", 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000243308 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.758827+0000", 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00083951299999999996 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.759986+0000", 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001159024 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.760836+0000", 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084967199999999999 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.760895+0000", 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9027999999999998e-05 2024-09-14T07:08:07.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.760947+0000", 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1715000000000002e-05 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4498.0:500 3.e e34/21 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:head v 34'4, mlcod=24'2)", 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:04.081823+0000", 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7211471949999999, 2024-09-14T07:08:07.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0099577490000000001, 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081823+0000", 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081823+0000", 2024-09-14T07:08:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081825+0000", 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0600000000000002e-06 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082583+0000", 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00075779099999999996 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082623+0000", 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9935999999999998e-05 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082643+0000", 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0143e-05 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082794+0000", 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000150552 2024-09-14T07:08:07.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082850+0000", 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5733000000000003e-05 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.091675+0000", 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0088256510000000003 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.091781+0000", 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105883 2024-09-14T07:08:07.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4498.0:501 3.e e34/21 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:head v 34'6, mlcod=24'2)", 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.717558023, 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0073848630000000002, 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4399999999999999e-07 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085422+0000", 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2439999999999993e-06 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085437+0000", 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5321000000000001e-05 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085445+0000", 2024-09-14T07:08:07.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0409999999999998e-06 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.090460+0000", 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.005015238 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.090494+0000", 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3476000000000003e-05 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.092716+0000", 2024-09-14T07:08:07.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022224020000000001 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.092797+0000", 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1297000000000004e-05 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "pg_update_log_missing(3.1b epoch 36/21 rep_tid 103 entries 36'49 (0'0) error 3:dfe84132:::rbd_data.1161a9b4f57.000000000000003c:head by client.4525.0:283 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'46)", 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.057226+0000", 2024-09-14T07:08:07.925 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.74574479599999999, 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066423860000000001, 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057226+0000", 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057226+0000", 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:07.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057227+0000", 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057238+0000", 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022000000000001e-05 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057273+0000", 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5244999999999999e-05 2024-09-14T07:08:07.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057295+0000", 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1693999999999999e-05 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057870+0000", 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057496999999999995 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:07.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063868+0000", 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0059983950000000001 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:07.929 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:08:07.931 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:07.931 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:07.930+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:08.031 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:08.030+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:08.124 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:08.124 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:08.124 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:08.124 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:08.132 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:08.130+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:08.139 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:08.234 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:08.230+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:08.262 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:08.262 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:08.262 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:08.262 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:08.262 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:08.276 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:08.334 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:08.330+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:08.359 INFO:teuthology.orchestra.run.smithi145.stdout:522 write 0x593400 thru 0x599bca (0x67cb bytes) 2024-09-14T07:08:08.360 INFO:teuthology.orchestra.run.smithi145.stdout:524 read 0x69b9800 thru 0x69c7694 (0xde95 bytes) 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:379 3.1d 3:b8c07fa4:::rbd_header.11611b5ca054:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.814369+0000", 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.587242842, 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010161081000000001, 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 379 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814369+0000", 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814369+0000", 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814370+0000", 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8369999999999999e-06 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814381+0000", 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0219e-05 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814411+0000", 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.029e-05 2024-09-14T07:08:08.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814426+0000", 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814751+0000", 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032523099999999999 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.814770+0000", 2024-09-14T07:08:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8425999999999999e-05 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.823545+0000", 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00877556 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.823750+0000", 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020448999999999999 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.824530+0000", 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00078010799999999995 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:919 3.1d 3:b8c07fa4:::rbd_header.11611b5ca054:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.815581+0000", 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.586030555000001, 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010542739000000001, 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 919 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815581+0000", 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815581+0000", 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815583+0000", 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815595+0000", 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1759999999999999e-05 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815633+0000", 2024-09-14T07:08:08.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8503e-05 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815656+0000", 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2507e-05 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815820+0000", 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000164291 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.815837+0000", 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6864e-05 2024-09-14T07:08:08.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.824604+0000", 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0087665460000000001 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.824811+0000", 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020722399999999999 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.826124+0000", 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001312864 2024-09-14T07:08:08.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.571074393, 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:08.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:08.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:08.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.570300045, 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:08.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:08.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:08.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:08.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.423 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:928 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [write 933888~49152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839499+0000", 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.562112397, 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017193320000000002, 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 928 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839499+0000", 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839499+0000", 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839500+0000", 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839653+0000", 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015312100000000001 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839672+0000", 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8629000000000001e-05 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839681+0000", 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8759999999999994e-06 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839746+0000", 2024-09-14T07:08:08.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5134000000000004e-05 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840343+0000", 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00059738 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845463+0000", 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051200509999999996 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845520+0000", 2024-09-14T07:08:08.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6552999999999999e-05 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845627+0000", 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106894 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849612+0000", 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039852749999999999 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856485+0000", 2024-09-14T07:08:08.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0068730379999999997 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856567+0000", 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2553999999999996e-05 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.856692+0000", 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012496299999999999 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1168 3.2 e26/21 3:42220d69:::rbd_data.11611b5ca054.000000000000002d:head v 26'9, mlcod=26'7)", 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.034190+0000", 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.367421887000001, 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013133113, 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034190+0000", 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034190+0000", 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034192+0000", 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.984e-06 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034237+0000", 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4963000000000003e-05 2024-09-14T07:08:08.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034273+0000", 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6423e-05 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034293+0000", 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9686000000000001e-05 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037078+0000", 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027857659999999998 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037132+0000", 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3535999999999997e-05 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047259+0000", 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010127038 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047323+0000", 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3717e-05 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.433 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1170 3.2 e26/21 3:40d5ec5b:::rbd_data.11611b5ca054.000000000000002f:head v 26'10, mlcod=26'7)", 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.037387+0000", 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.364224236, 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010559537000000001, 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037387+0000", 2024-09-14T07:08:08.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037387+0000", 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037389+0000", 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.68e-06 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037489+0000", 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100574 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037517+0000", 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7185000000000001e-05 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.037553+0000", 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6714000000000001e-05 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040335+0000", 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027815140000000001 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040380+0000", 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4965000000000001e-05 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047842+0000", 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0074618490000000004 2024-09-14T07:08:08.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.047947+0000", 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010505600000000001 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4498.0:500 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:04.080119+0000", 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3214926880000002, 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011955844, 2024-09-14T07:08:08.438 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4498", 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 500 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.080119+0000", 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.080119+0000", 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.080121+0000", 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4260000000000002e-06 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081185+0000", 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010641260000000001 2024-09-14T07:08:08.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081215+0000", 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9822999999999999e-05 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081236+0000", 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0718999999999999e-05 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081313+0000", 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6682999999999997e-05 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081430+0000", 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116981 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081658+0000", 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022809600000000001 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085987+0000", 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043288729999999996 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.091960+0000", 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0059738049999999996 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.092026+0000", 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5302000000000006e-05 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.092075+0000", 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.901e-05 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4498.0:501 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:head [write 0~0] snapc 6=[6] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:04.081282+0000", 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3203298339999998, 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011795081000000001, 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4498", 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 501 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081282+0000", 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081282+0000", 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081284+0000", 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8899999999999999e-06 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081291+0000", 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4070000000000004e-06 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081335+0000", 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.384e-05 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081352+0000", 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7626e-05 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.084913+0000", 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035607149999999999 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085009+0000", 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5956000000000001e-05 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085220+0000", 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000211392 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.086769+0000", 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015486600000000001 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.092982+0000", 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0062126760000000003 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.093035+0000", 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2933000000000003e-05 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.093077+0000", 2024-09-14T07:08:08.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1986000000000001e-05 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:344 3.c 3:376c12bc:::rbd_data.1161a9b4f57.0000000000000062:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.058517+0000", 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.343094531, 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0098127040000000002, 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.450 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 344 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058517+0000", 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058517+0000", 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058518+0000", 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.37e-07 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058521+0000", 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4410000000000002e-06 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058531+0000", 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8660000000000001e-06 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.058539+0000", 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6070000000000002e-06 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061158+0000", 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002619793 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.061325+0000", 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016645599999999999 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068330+0000", 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070048039999999999 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.454 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:372 3.16 3:6da1aa4c:::rbd_data.1161a9b4f57.0000000000000037:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.063691+0000", 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.337920233, 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.01700552, 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 372 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063691+0000", 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063691+0000", 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063693+0000", 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.31e-06 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063698+0000", 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063711+0000", 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2254e-05 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063718+0000", 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8010000000000004e-06 2024-09-14T07:08:08.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063778+0000", 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9370000000000002e-05 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063856+0000", 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8053000000000006e-05 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.064053+0000", 2024-09-14T07:08:08.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019726499999999999 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.064993+0000", 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00093961499999999998 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.080588+0000", 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015595295 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.080658+0000", 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9851000000000006e-05 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.080697+0000", 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8992999999999997e-05 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:380 3.f 3:f7d7ad2b:::rbd_data.1161a9b4f57.0000000000000061:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.460 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.065332+0000", 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.336279513, 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.02399136, 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 380 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065332+0000", 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065332+0000", 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065333+0000", 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065342+0000", 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8319999999999995e-06 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065351+0000", 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4210000000000001e-06 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065359+0000", 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3839999999999999e-06 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065469+0000", 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011014999999999999 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065543+0000", 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3898000000000003e-05 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065721+0000", 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000177992 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067266+0000", 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015454519999999999 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089217+0000", 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021950509 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089281+0000", 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4144000000000001e-05 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089323+0000", 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2633000000000003e-05 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:381 3.16 3:6accdcd0:::rbd_data.1161a9b4f57.0000000000000043:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.065381+0000", 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.3362301560000001, 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.025427293, 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.466 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 381 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065381+0000", 2024-09-14T07:08:08.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065381+0000", 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065382+0000", 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2600000000000002e-07 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065386+0000", 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8190000000000002e-06 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065393+0000", 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5689999999999999e-06 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065400+0000", 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0979999999999999e-06 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.065931+0000", 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00053180999999999997 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066010+0000", 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8738000000000004e-05 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066204+0000", 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000194112 2024-09-14T07:08:08.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067744+0000", 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015402199999999999 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090713+0000", 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022969026999999999 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090770+0000", 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6422000000000003e-05 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090809+0000", 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8751999999999998e-05 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:384 3.16 3:692c0166:::rbd_data.1161a9b4f57.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.066099+0000", 2024-09-14T07:08:08.472 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.335512708, 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.024835596000000001, 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 384 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066099+0000", 2024-09-14T07:08:08.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066099+0000", 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066100+0000", 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7700000000000004e-07 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066107+0000", 2024-09-14T07:08:08.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1999999999999997e-06 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066115+0000", 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9780000000000005e-06 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066121+0000", 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3239999999999999e-06 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066457+0000", 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033628199999999998 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066526+0000", 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8590999999999994e-05 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066702+0000", 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017597500000000001 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067750+0000", 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001047704 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090866+0000", 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.023116385 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090898+0000", 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2166999999999999e-05 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090934+0000", 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6213e-05 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:385 3.16 3:6dc22c2a:::rbd_data.1161a9b4f57.0000000000000057:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.066133+0000", 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.3354785790000001, 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.024935839000000001, 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:08.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 385 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066133+0000", 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066133+0000", 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066133+0000", 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.75e-07 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066137+0000", 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0589999999999998e-06 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066143+0000", 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.409e-06 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.066148+0000", 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3449999999999996e-06 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067276+0000", 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001127812 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067347+0000", 2024-09-14T07:08:08.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1085000000000001e-05 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067526+0000", 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000178593 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.071124+0000", 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003597874 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.090973+0000", 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.019848991 2024-09-14T07:08:08.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091024+0000", 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1789999999999997e-05 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091069+0000", 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4305999999999999e-05 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:401 3.16 3:6a554657:::rbd_data.1161a9b4f57.0000000000000069:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.067219+0000", 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.3343923229999999, 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.024044875, 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 401 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067219+0000", 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067219+0000", 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067220+0000", 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5300000000000002e-07 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067225+0000", 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067235+0000", 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8970000000000003e-06 2024-09-14T07:08:08.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067243+0000", 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0600000000000008e-06 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.067930+0000", 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00068693900000000002 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068045+0000", 2024-09-14T07:08:08.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011487100000000001 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091264+0000", 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.023218882999999999 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:410 3.16 3:6c478345:::rbd_data.1161a9b4f57.000000000000006c:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.068841+0000", 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.3327699850000001, 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022555861999999999, 2024-09-14T07:08:08.489 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 410 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068841+0000", 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068841+0000", 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068843+0000", 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068848+0000", 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1830000000000002e-06 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068861+0000", 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2799999999999999e-05 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.068887+0000", 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6525000000000001e-05 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.069260+0000", 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037280000000000001 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.069394+0000", 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013450100000000001 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091397+0000", 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022002931999999999 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:448 3.16 3:6ad7c179:::rbd_data.1161a9b4f57.0000000000000079:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.073302+0000", 2024-09-14T07:08:08.493 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.328309218, 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.018251186999999999, 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 448 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073302+0000", 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073302+0000", 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073303+0000", 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3699999999999999e-07 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073308+0000", 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0239999999999997e-06 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073317+0000", 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.363e-06 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073325+0000", 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6380000000000004e-06 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.074159+0000", 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00083391400000000005 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.074316+0000", 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015735799999999999 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091553+0000", 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017237052999999999 2024-09-14T07:08:08.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:457 3.16 3:6b0d038f:::rbd_data.1161a9b4f57.000000000000007d:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.073885+0000", 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.327726628, 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.017800044000000001, 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 457 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073885+0000", 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073885+0000", 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073886+0000", 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2399999999999997e-07 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073890+0000", 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.42e-06 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073898+0000", 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7519999999999996e-06 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.073904+0000", 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.896e-06 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.074797+0000", 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00089298100000000005 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.074982+0000", 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018527699999999999 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.091685+0000", 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016702893999999999 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.502 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4525.0:473 3.f 3:f598cede:::rbd_data.1161a9b4f57.0000000000000085:head [stat,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.078319+0000", 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.323292879, 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011276556, 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4525", 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 473 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:08.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078319+0000", 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078319+0000", 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078320+0000", 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.879999999999999e-07 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078325+0000", 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1410000000000004e-06 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078339+0000", 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3793e-05 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078348+0000", 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4110000000000002e-06 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:08.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078412+0000", 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3802999999999999e-05 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.078543+0000", 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131533 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.089595+0000", 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011051887 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:08.506 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:08.507 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:08.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:08.509 INFO:teuthology.orchestra.run.smithi145.stdout:525 write 0xebd5000 thru 0xebdadce (0x5dcf bytes) 2024-09-14T07:08:08.509 INFO:teuthology.orchestra.run.smithi145.stdout:526 punch from 0x1d6d800 to 0x1d7a200, (0xca00 bytes) 2024-09-14T07:08:08.509 INFO:teuthology.orchestra.run.smithi145.stdout:530 read 0xd620200 thru 0xd62e5b8 (0xe3b9 bytes) 2024-09-14T07:08:08.509 INFO:teuthology.orchestra.run.smithi145.stdout:531 punch from 0x47c6e00 to 0x47d3e00, (0xd000 bytes) 2024-09-14T07:08:08.509 INFO:teuthology.orchestra.run.smithi145.stdout:532 trunc from 0xebdae00 to 0xc40d200 2024-09-14T07:08:08.510 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:08.506+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:08.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:08.543 INFO:teuthology.orchestra.run.smithi145.stdout:535 read 0xa554400 thru 0xa55b575 (0x7176 bytes) 2024-09-14T07:08:08.545 INFO:teuthology.orchestra.run.smithi145.stdout:536 write 0x8daac00 thru 0x8db6c1d (0xc01e bytes) 2024-09-14T07:08:08.545 INFO:teuthology.orchestra.run.smithi145.stdout:537 read 0x9b46400 thru 0x9b4d5c0 (0x71c1 bytes) 2024-09-14T07:08:08.545 INFO:teuthology.orchestra.run.smithi145.stdout:539 read 0xaa9d600 thru 0xaaa3a31 (0x6432 bytes) 2024-09-14T07:08:08.578 INFO:teuthology.orchestra.run.smithi145.stdout:541 trunc from 0xc40d200 to 0x5cbf800 2024-09-14T07:08:08.609 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:08.606+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:08.662 INFO:teuthology.orchestra.run.smithi145.stdout:542 read 0x1826400 thru 0x1835282 (0xee83 bytes) 2024-09-14T07:08:08.666 INFO:teuthology.orchestra.run.smithi145.stdout:547 write 0x32a9200 thru 0x32ae5a2 (0x53a3 bytes) 2024-09-14T07:08:08.667 INFO:teuthology.orchestra.run.smithi145.stdout:550 flatten 2024-09-14T07:08:08.678 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:08.698 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:08.703 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:08.703 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:08.709 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:08.706+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:08.718 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:08.719 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:08.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:08.746 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:08.770 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:08.788 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:08.789 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:08.810 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:08.806+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:08.872 INFO:teuthology.orchestra.run.smithi145.stdout:552 read 0x18a3400 thru 0x18afcbc (0xc8bd bytes) 2024-09-14T07:08:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:08.880 INFO:teuthology.orchestra.run.smithi145.stdout:553 trunc from 0x5cbf800 to 0x772e400 2024-09-14T07:08:08.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:08.896 INFO:teuthology.orchestra.run.smithi145.stdout:554 read 0x6c38400 thru 0x6c3a078 (0x1c79 bytes) 2024-09-14T07:08:08.897 INFO:teuthology.orchestra.run.smithi145.stdout:556 read 0x2ed1600 thru 0x2ed4bda (0x35db bytes) 2024-09-14T07:08:08.938 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:08.938+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:08.953 INFO:teuthology.orchestra.run.smithi145.stdout:557 trunc from 0x772e400 to 0x155b000 2024-09-14T07:08:09.013 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:09.010+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:09.024 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:09.024 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:09.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:09.054 INFO:teuthology.orchestra.run.smithi145.stdout:560 read 0xaf1000 thru 0xaf8240 (0x7241 bytes) 2024-09-14T07:08:09.092 INFO:teuthology.orchestra.run.smithi145.stdout:561 trunc from 0x155b000 to 0xe838200 2024-09-14T07:08:09.114 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:09.110+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:09.115 INFO:teuthology.orchestra.run.smithi145.stdout:562 read 0x512ac00 thru 0x5130b63 (0x5f64 bytes) 2024-09-14T07:08:09.123 INFO:teuthology.orchestra.run.smithi145.stdout:563 trunc from 0xe838200 to 0xbb82a00 2024-09-14T07:08:09.148 INFO:teuthology.orchestra.run.smithi145.stdout:564 punch from 0x6802200 to 0x680b000, (0x8e00 bytes) 2024-09-14T07:08:09.149 INFO:teuthology.orchestra.run.smithi145.stdout:566 punch from 0x1d75e00 to 0x1d79c00, (0x3e00 bytes) 2024-09-14T07:08:09.149 INFO:teuthology.orchestra.run.smithi145.stdout:567 write 0x75b4a00 thru 0x75b851c (0x3b1d bytes) 2024-09-14T07:08:09.150 INFO:teuthology.orchestra.run.smithi145.stdout:568 read 0xd3400 thru 0xd70f5 (0x3cf6 bytes) 2024-09-14T07:08:09.150 INFO:teuthology.orchestra.run.smithi145.stdout:569 read 0x9e21200 thru 0x9e2973a (0x853b bytes) 2024-09-14T07:08:09.150 INFO:teuthology.orchestra.run.smithi145.stdout:570 punch from 0xafdba00 to 0xafe0800, (0x4e00 bytes) 2024-09-14T07:08:09.151 INFO:teuthology.orchestra.run.smithi145.stdout:571 read 0x549ea00 thru 0x54ac55e (0xdb5f bytes) 2024-09-14T07:08:09.151 INFO:teuthology.orchestra.run.smithi145.stdout:572 punch from 0x4b6ae00 to 0x4b70000, (0x5200 bytes) 2024-09-14T07:08:09.151 INFO:teuthology.orchestra.run.smithi145.stdout:573 read 0x517d200 thru 0x5188dce (0xbbcf bytes) 2024-09-14T07:08:09.152 INFO:teuthology.orchestra.run.smithi145.stdout:574 punch from 0x52c4a00 to 0x52d2e00, (0xe400 bytes) 2024-09-14T07:08:09.152 INFO:teuthology.orchestra.run.smithi145.stdout:575 read 0xb121e00 thru 0xb1225e6 (0x7e7 bytes) 2024-09-14T07:08:09.152 INFO:teuthology.orchestra.run.smithi145.stdout:576 read 0x96a1000 thru 0x96af4ba (0xe4bb bytes) 2024-09-14T07:08:09.152 INFO:teuthology.orchestra.run.smithi145.stdout:577 write 0x215e600 thru 0x21615b9 (0x2fba bytes) 2024-09-14T07:08:09.152 INFO:teuthology.orchestra.run.smithi145.stdout:578 clone 6 order 19 su 32768 sc 7 2024-09-14T07:08:09.215 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:09.214+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:09.220 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:09.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:09.315 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:09.314+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:09.402 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:09.402 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:09.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:09.417 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:09.414+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:09.517 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:09.514+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:09.587 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:09.587 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:09.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:09.647 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:09.646+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:09.719 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:09.718+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:09.767 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:09.767 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:09.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:09.822 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:09.822+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:09.922 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:09.922+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:09.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:08:09.959 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:09.959 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:09.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:10.022 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:10.018+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:10.124 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:10.122+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:10.155 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:10.155 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:10.170 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:10.225 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:10.222+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:10.305 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:10.306 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:10.321 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:10.326 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:10.326+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:10.427 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:10.426+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:10.492 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:10.492 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:10.507 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:10.530 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:10.526+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:10.621 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T07:08:10.630 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:10.630 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:10.630 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:10.626+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:10.644 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:10.735 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:10.734+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:10.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:10.808 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:10.808 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:10.821 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:10.858 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:10.858+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:10.933 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:10.930+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:08:10.976 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:10.995 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:10.997 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:10.997 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:11.012 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:11.013 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:11.034 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:11.035 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:11.030+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:11.051 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:11.070 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:11.070 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:11.087 INFO:teuthology.orchestra.run.smithi145.stdout:579 write 0x5a70800 thru 0x5a7284d (0x204e bytes) 2024-09-14T07:08:11.088 INFO:teuthology.orchestra.run.smithi145.stdout:580 write 0x6fe0e00 thru 0x6fe23f1 (0x15f2 bytes) 2024-09-14T07:08:11.109 INFO:teuthology.orchestra.run.smithi145.stdout:581 trunc from 0xbb82a00 to 0x955d800 2024-09-14T07:08:11.134 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:11.130+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:11.155 INFO:teuthology.orchestra.run.smithi145.stdout:585 write 0x940b800 thru 0x94188fc (0xd0fd bytes) 2024-09-14T07:08:11.157 INFO:teuthology.orchestra.run.smithi145.stdout:587 read 0x54c6400 thru 0x54d558d (0xf18e bytes) 2024-09-14T07:08:11.171 INFO:teuthology.orchestra.run.smithi145.stdout:589 trunc from 0x955d800 to 0xc854000 2024-09-14T07:08:11.186 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:11.186 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:11.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:11.186 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:11.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.186 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-14T07:08:11.186 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:08:11.187 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.388350324000001, 2024-09-14T07:08:11.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070947240000000002, 2024-09-14T07:08:11.187 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.187 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.187 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:11.188 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:11.188 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 7 2024-09-14T07:08:11.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.188 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:08:11.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793550+0000", 2024-09-14T07:08:11.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793553+0000", 2024-09-14T07:08:11.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3730000000000001e-06 2024-09-14T07:08:11.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793583+0000", 2024-09-14T07:08:11.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9675999999999998e-05 2024-09-14T07:08:11.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793621+0000", 2024-09-14T07:08:11.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7370000000000003e-05 2024-09-14T07:08:11.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793647+0000", 2024-09-14T07:08:11.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6276999999999999e-05 2024-09-14T07:08:11.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793835+0000", 2024-09-14T07:08:11.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018812500000000001 2024-09-14T07:08:11.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:11.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.793852+0000", 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.734e-05 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797434+0000", 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035819189999999998 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797559+0000", 2024-09-14T07:08:11.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012508500000000001 2024-09-14T07:08:11.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:11.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.797726+0000", 2024-09-14T07:08:11.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000166797 2024-09-14T07:08:11.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.798685+0000", 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00095928199999999999 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800546+0000", 2024-09-14T07:08:11.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018609189999999999 2024-09-14T07:08:11.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800602+0000", 2024-09-14T07:08:11.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5365999999999998e-05 2024-09-14T07:08:11.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:48.800645+0000", 2024-09-14T07:08:11.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3195000000000002e-05 2024-09-14T07:08:11.199 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.199 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-14T07:08:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:08:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.380916812999999, 2024-09-14T07:08:11.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0097102179999999996, 2024-09-14T07:08:11.201 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.201 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.201 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:11.202 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:11.202 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9 2024-09-14T07:08:11.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.202 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:08:11.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800984+0000", 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.800987+0000", 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801000+0000", 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2897e-05 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801028+0000", 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8575999999999999e-05 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801054+0000", 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6367e-05 2024-09-14T07:08:11.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801282+0000", 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022778100000000001 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.801300+0000", 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7237000000000001e-05 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808053+0000", 2024-09-14T07:08:11.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0067535249999999998 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808316+0000", 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000263224 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.808688+0000", 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037128699999999998 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810367+0000", 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001679151 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810427+0000", 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0338999999999999e-05 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810572+0000", 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014462200000000001 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:49.810694+0000", 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000122192 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:929 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.342330060999998, 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013302158999999999, 2024-09-14T07:08:11.209 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 929 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839570+0000", 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839572+0000", 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6780000000000001e-06 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839580+0000", 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0989999999999999e-06 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839590+0000", 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839598+0000", 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3699999999999997e-06 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840130+0000", 2024-09-14T07:08:11.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00053217 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840818+0000", 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00068814799999999997 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845641+0000", 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048226160000000001 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.845730+0000", 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9687999999999999e-05 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846136+0000", 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000405583 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849695+0000", 2024-09-14T07:08:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00355887 2024-09-14T07:08:11.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852725+0000", 2024-09-14T07:08:11.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030301009999999999 2024-09-14T07:08:11.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852786+0000", 2024-09-14T07:08:11.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1081999999999996e-05 2024-09-14T07:08:11.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852873+0000", 2024-09-14T07:08:11.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6608999999999997e-05 2024-09-14T07:08:11.217 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.217 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.217 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:930 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.342290478999999, 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013501599, 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.218 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 930 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839610+0000", 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839613+0000", 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4169999999999999e-06 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839618+0000", 2024-09-14T07:08:11.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1499999999999998e-06 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839622+0000", 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.631e-06 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841189+0000", 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001567768 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:11.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841262+0000", 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2105000000000002e-05 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846495+0000", 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052335999999999997 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846559+0000", 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3609999999999996e-05 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.846782+0000", 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022373299999999999 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849705+0000", 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002922998 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852940+0000", 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032343070000000001 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853014+0000", 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3967999999999994e-05 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853112+0000", 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7806000000000005e-05 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.224 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:931 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.342222174, 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.01360341, 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 931 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839678+0000", 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839679+0000", 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7299999999999996e-07 2024-09-14T07:08:11.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839682+0000", 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839688+0000", 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9569999999999999e-06 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839693+0000", 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6289999999999998e-06 2024-09-14T07:08:11.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841324+0000", 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016306090000000001 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841425+0000", 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010129299999999999 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847149+0000", 2024-09-14T07:08:11.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057235139999999999 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847194+0000", 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5117000000000002e-05 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847441+0000", 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000247428 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852669+0000", 2024-09-14T07:08:11.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052276029999999999 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853162+0000", 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049292300000000004 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853202+0000", 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0525999999999998e-05 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853282+0000", 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9513999999999998e-05 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:932 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.342115352, 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013718054, 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.231 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 932 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839785+0000", 2024-09-14T07:08:11.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839786+0000", 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4499999999999996e-07 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839794+0000", 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6450000000000005e-06 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839805+0000", 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1489e-05 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.839814+0000", 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6540000000000005e-06 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841579+0000", 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017656480000000001 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:11.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841669+0000", 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9806e-05 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.847963+0000", 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0062937080000000003 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848135+0000", 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017191299999999999 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.848791+0000", 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00065584300000000003 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853128+0000", 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043374149999999998 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853351+0000", 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000223408 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853408+0000", 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6922000000000002e-05 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853503+0000", 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4758000000000003e-05 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:933 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.237 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.341684398000002, 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015741667000000001, 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 933 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840216+0000", 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840217+0000", 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1909999999999999e-06 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840230+0000", 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3226e-05 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840244+0000", 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3108999999999999e-05 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.840252+0000", 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2910000000000007e-06 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841706+0000", 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001454056 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841766+0000", 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0494000000000001e-05 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.849824+0000", 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080573079999999991 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850028+0000", 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020432999999999999 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.850748+0000", 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072041399999999995 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853135+0000", 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002386885 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855783+0000", 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026479350000000001 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855851+0000", 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7534999999999996e-05 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.855958+0000", 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010689299999999999 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:935 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.339975259999999, 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016747201, 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.244 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 935 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841925+0000", 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841926+0000", 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841932+0000", 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2099999999999998e-06 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841943+0000", 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0981e-05 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.841950+0000", 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842004+0000", 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4422000000000002e-05 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842064+0000", 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9967999999999999e-05 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851737+0000", 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0096733600000000006 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.851825+0000", 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7855999999999994e-05 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.852258+0000", 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000432383 2024-09-14T07:08:11.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853520+0000", 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001262787 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858562+0000", 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050418269999999996 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858623+0000", 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0789999999999999e-05 2024-09-14T07:08:11.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858672+0000", 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9348999999999998e-05 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:936 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.339733502000001, 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.018202472000000001, 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.250 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 936 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842167+0000", 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842168+0000", 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3300000000000002e-07 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842170+0000", 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6539999999999999e-06 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842178+0000", 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.413e-06 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842182+0000", 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3139999999999997e-06 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842231+0000", 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8553000000000001e-05 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.842290+0000", 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9534000000000003e-05 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853603+0000", 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011312579 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.853796+0000", 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299000000000001 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.854842+0000", 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00104612 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.858690+0000", 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038478589999999999 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860251+0000", 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015616829999999999 2024-09-14T07:08:11.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860315+0000", 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3587e-05 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.860369+0000", 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4552999999999998e-05 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1142 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.164855304, 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.007319173, 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1142 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017045+0000", 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017046+0000", 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:08:11.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017061+0000", 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.467e-05 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017076+0000", 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5421e-05 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017088+0000", 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1287999999999999e-05 2024-09-14T07:08:11.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019464+0000", 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023761590000000001 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.019645+0000", 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000180772 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.020444+0000", 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00079905700000000004 2024-09-14T07:08:11.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023077+0000", 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002633093 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024178+0000", 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011013850000000001 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024296+0000", 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117586 2024-09-14T07:08:11.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024364+0000", 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8585999999999997e-05 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1145 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.164556984000001, 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077736239999999998, 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.262 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1145 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017343+0000", 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017346+0000", 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2369999999999999e-06 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017358+0000", 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2602999999999999e-05 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017390+0000", 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.188e-05 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.017413+0000", 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.243e-05 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021308+0000", 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038951799999999998 2024-09-14T07:08:11.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.021511+0000", 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020368099999999999 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.022262+0000", 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00075023799999999997 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.023822+0000", 2024-09-14T07:08:11.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015606750000000001 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025018+0000", 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001195149 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8764999999999997e-05 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025117+0000", 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0785999999999999e-05 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1159 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.15693156, 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0074537930000000002, 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1159 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024969+0000", 2024-09-14T07:08:11.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024970+0000", 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5939999999999999e-06 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.024981+0000", 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0145e-05 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025004+0000", 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3468999999999998e-05 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025021+0000", 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6674e-05 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025194+0000", 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173721 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025445+0000", 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025037099999999999 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.026330+0000", 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000885073 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032132+0000", 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058019819999999998 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032250+0000", 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118447 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032351+0000", 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100966 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032423+0000", 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1351000000000002e-05 2024-09-14T07:08:11.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1160 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.156825627, 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0076908269999999999, 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1160 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025075+0000", 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025076+0000", 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.251e-06 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025084+0000", 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5689999999999999e-06 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025102+0000", 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8734000000000001e-05 2024-09-14T07:08:11.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025116+0000", 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3465999999999999e-05 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027183+0000", 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002066877 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.027355+0000", 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017235800000000001 2024-09-14T07:08:11.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028074+0000", 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00071921500000000005 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032586+0000", 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045120960000000002 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032617+0000", 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0468999999999999e-05 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032697+0000", 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9939999999999997e-05 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032766+0000", 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8851999999999997e-05 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1161 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.156727421999999, 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0078300520000000005, 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1161 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.280 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025173+0000", 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025174+0000", 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4139999999999999e-06 2024-09-14T07:08:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025184+0000", 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3489999999999998e-06 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025201+0000", 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7011e-05 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025214+0000", 2024-09-14T07:08:11.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3237e-05 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.028886+0000", 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036715649999999999 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029039+0000", 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015298300000000001 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.029611+0000", 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057197599999999997 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032782+0000", 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031712340000000002 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032863+0000", 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0907000000000003e-05 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.032957+0000", 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4715000000000004e-05 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033003+0000", 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5661000000000002e-05 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:1162 3.9 3:92b5a216:::rbd_object_map.11611b5ca054:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.15665976, 2024-09-14T07:08:11.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084900349999999999, 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1162 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025241+0000", 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025242+0000", 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3829999999999999e-06 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025248+0000", 2024-09-14T07:08:11.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1999999999999999e-06 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025283+0000", 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4347000000000002e-05 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.025295+0000", 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2268e-05 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030146+0000", 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048513200000000001 2024-09-14T07:08:11.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.030309+0000", 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000163225 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.031281+0000", 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00097148600000000003 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033053+0000", 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001771941 2024-09-14T07:08:11.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033582+0000", 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052929400000000003 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033653+0000", 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1285999999999995e-05 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.033731+0000", 2024-09-14T07:08:11.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7285000000000003e-05 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4406.0:1167 3.a e26/21 3:509716c6:::rbd_data.11611b5ca054.000000000000002a:head v 26'12, mlcod=26'10)", 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.147787170000001, 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065049039999999997, 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034113+0000", 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034114+0000", 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0800000000000003e-07 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034143+0000", 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8532000000000001e-05 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034158+0000", 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5525000000000001e-05 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034169+0000", 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0726e-05 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034296+0000", 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012745700000000001 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.034352+0000", 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5229999999999998e-05 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040532+0000", 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0061802130000000004 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:54.040618+0000", 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6312999999999998e-05 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4539.0:2027 3.1a 3:59c1d47c:::rbd_header.116147128396:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:57.754109+0000", 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.427791114, 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006837419, 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.295 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4539", 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2491585557", 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2027 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754109+0000", 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754109+0000", 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754113+0000", 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.32e-06 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754129+0000", 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6160000000000001e-05 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754169+0000", 2024-09-14T07:08:11.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0583000000000001e-05 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754195+0000", 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5673e-05 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754283+0000", 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8084999999999994e-05 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.754288+0000", 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6110000000000004e-06 2024-09-14T07:08:11.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.757744+0000", 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034567270000000002 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.757988+0000", 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000243308 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:11.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.758827+0000", 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00083951299999999996 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.759986+0000", 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001159024 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.760836+0000", 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084967199999999999 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.760895+0000", 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9027999999999998e-05 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:57.760947+0000", 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1715000000000002e-05 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4498.0:500 3.e e34/21 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:head v 34'4, mlcod=24'2)", 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:04.081823+0000", 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.1000770119999999, 2024-09-14T07:08:11.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0099577490000000001, 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081823+0000", 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081823+0000", 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.081825+0000", 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0600000000000002e-06 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082583+0000", 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00075779099999999996 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082623+0000", 2024-09-14T07:08:11.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9935999999999998e-05 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082643+0000", 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0143e-05 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082794+0000", 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000150552 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.082850+0000", 2024-09-14T07:08:11.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5733000000000003e-05 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.091675+0000", 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0088256510000000003 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.091781+0000", 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105883 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4498.0:501 3.e e34/21 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:head v 34'6, mlcod=24'2)", 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.09648784, 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0073848630000000002, 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085413+0000", 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4399999999999999e-07 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085422+0000", 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2439999999999993e-06 2024-09-14T07:08:11.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085437+0000", 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5321000000000001e-05 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.085445+0000", 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0409999999999998e-06 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.090460+0000", 2024-09-14T07:08:11.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.005015238 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.090494+0000", 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3476000000000003e-05 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.092716+0000", 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022224020000000001 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:04.092797+0000", 2024-09-14T07:08:11.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1297000000000004e-05 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "pg_update_log_missing(3.1b epoch 36/21 rep_tid 103 entries 36'49 (0'0) error 3:dfe84132:::rbd_data.1161a9b4f57.000000000000003c:head by client.4525.0:283 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'46)", 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:07.057226+0000", 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.1246746129999998, 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066423860000000001, 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057226+0000", 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057226+0000", 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057227+0000", 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057238+0000", 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022000000000001e-05 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057273+0000", 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5244999999999999e-05 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057295+0000", 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1693999999999999e-05 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.057870+0000", 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057496999999999995 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:07.063868+0000", 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0059983950000000001 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.313 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:11.315 INFO:teuthology.orchestra.run.smithi145.stdout:590 trunc from 0xc854000 to 0x6164600 2024-09-14T07:08:11.315 INFO:teuthology.orchestra.run.smithi145.stdout:593 read 0x15bce00 thru 0x15c4a03 (0x7c04 bytes) 2024-09-14T07:08:11.315 INFO:teuthology.orchestra.run.smithi145.stdout:597 read 0x48e3200 thru 0x48e3cb6 (0xab7 bytes) 2024-09-14T07:08:11.315 INFO:teuthology.orchestra.run.smithi145.stdout:599 read 0x25a1200 thru 0x25afe27 (0xec28 bytes) 2024-09-14T07:08:11.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:11.316 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:11.314+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:11.340 INFO:teuthology.orchestra.run.smithi145.stdout:601 write 0xad66200 thru 0xad69ec0 (0x3cc1 bytes) 2024-09-14T07:08:11.396 INFO:teuthology.orchestra.run.smithi145.stdout:603 write 0xdb16800 thru 0xdb1a774 (0x3f75 bytes) 2024-09-14T07:08:11.417 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:11.414+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:11.442 INFO:teuthology.orchestra.run.smithi145.stdout:605 write 0xe5fd400 thru 0xe605de9 (0x89ea bytes) 2024-09-14T07:08:11.444 INFO:teuthology.orchestra.run.smithi145.stdout:606 write 0x3807a00 thru 0x38120a0 (0xa6a1 bytes) 2024-09-14T07:08:11.444 INFO:teuthology.orchestra.run.smithi145.stdout:607 write 0xdd2f200 thru 0xdd37f37 (0x8d38 bytes) 2024-09-14T07:08:11.451 INFO:teuthology.orchestra.run.smithi145.stdout:608 trunc from 0xe605e00 to 0xbdbba00 2024-09-14T07:08:11.494 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:11.494 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:11.494 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:11.494 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:11.509 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:11.511 INFO:teuthology.orchestra.run.smithi145.stdout:610 read 0x167ea00 thru 0x1686dba (0x83bb bytes) 2024-09-14T07:08:11.514 INFO:teuthology.orchestra.run.smithi145.stdout:611 write 0xbc0a00 thru 0xbc6a1d (0x601e bytes) 2024-09-14T07:08:11.516 INFO:teuthology.orchestra.run.smithi145.stdout:612 read 0x89cf800 thru 0x89de1fc (0xe9fd bytes) 2024-09-14T07:08:11.516 INFO:teuthology.orchestra.run.smithi145.stdout:613 read 0x44ff600 thru 0x45094d2 (0x9ed3 bytes) 2024-09-14T07:08:11.516 INFO:teuthology.orchestra.run.smithi145.stdout:615 write 0x1caa600 thru 0x1cb47cf (0xa1d0 bytes) 2024-09-14T07:08:11.517 INFO:teuthology.orchestra.run.smithi145.stdout:619 read 0xea0800 thru 0xea5a34 (0x5235 bytes) 2024-09-14T07:08:11.519 INFO:teuthology.orchestra.run.smithi145.stdout:621 punch from 0x38ae800 to 0x38b8000, (0x9800 bytes) 2024-09-14T07:08:11.520 INFO:teuthology.orchestra.run.smithi145.stdout:623 read 0x83f6000 thru 0x8403673 (0xd674 bytes) 2024-09-14T07:08:11.520 INFO:teuthology.orchestra.run.smithi145.stdout:627 read 0x9858400 thru 0x986779b (0xf39c bytes) 2024-09-14T07:08:11.521 INFO:teuthology.orchestra.run.smithi145.stdout:628 write 0xa98c800 thru 0xa99b685 (0xee86 bytes) 2024-09-14T07:08:11.521 INFO:teuthology.orchestra.run.smithi145.stdout:629 punch from 0x470be00 to 0x470ec00, (0x2e00 bytes) 2024-09-14T07:08:11.522 INFO:teuthology.orchestra.run.smithi145.stdout:631 read 0x9c39e00 thru 0x9c456ce (0xb8cf bytes) 2024-09-14T07:08:11.523 INFO:teuthology.orchestra.run.smithi145.stdout:632 read 0x7d65000 thru 0x7d6952b (0x452c bytes) 2024-09-14T07:08:11.523 INFO:teuthology.orchestra.run.smithi145.stdout:634 read 0xf68a00 thru 0xf6ae38 (0x2439 bytes) 2024-09-14T07:08:11.529 INFO:teuthology.orchestra.run.smithi145.stdout:636 read 0x9804400 thru 0x9807659 (0x325a bytes) 2024-09-14T07:08:11.529 INFO:teuthology.orchestra.run.smithi145.stdout:637 write 0xbd82800 thru 0xbd84bc6 (0x23c7 bytes) 2024-09-14T07:08:11.529 INFO:teuthology.orchestra.run.smithi145.stdout:638 punch from 0x1af3200 to 0x1af8000, (0x4e00 bytes) 2024-09-14T07:08:11.530 INFO:teuthology.orchestra.run.smithi145.stdout:639 read 0x86c6400 thru 0x86d1a5c (0xb65d bytes) 2024-09-14T07:08:11.530 INFO:teuthology.orchestra.run.smithi145.stdout:640 punch from 0x8008e00 to 0x800b400, (0x2600 bytes) 2024-09-14T07:08:11.551 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:11.550+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:11.577 INFO:teuthology.orchestra.run.smithi145.stdout:642 write 0xdc6c000 thru 0xdc6de60 (0x1e61 bytes) 2024-09-14T07:08:11.579 INFO:teuthology.orchestra.run.smithi145.stdout:643 read 0x7244c00 thru 0x72526ba (0xdabb bytes) 2024-09-14T07:08:11.579 INFO:teuthology.orchestra.run.smithi145.stdout:644 read 0x5791200 thru 0x579551f (0x4320 bytes) 2024-09-14T07:08:11.579 INFO:teuthology.orchestra.run.smithi145.stdout:645 read 0x2cc0e00 thru 0x2cc69da (0x5bdb bytes) 2024-09-14T07:08:11.580 INFO:teuthology.orchestra.run.smithi145.stdout:646 punch from 0x54bd800 to 0x54c2200, (0x4a00 bytes) 2024-09-14T07:08:11.582 INFO:teuthology.orchestra.run.smithi145.stdout:648 write 0xbdff600 thru 0xbe09d4a (0xa74b bytes) 2024-09-14T07:08:11.582 INFO:teuthology.orchestra.run.smithi145.stdout:649 punch from 0x47d5200 to 0x47d7e00, (0x2c00 bytes) 2024-09-14T07:08:11.583 INFO:teuthology.orchestra.run.smithi145.stdout:650 write 0x182ba00 thru 0x182e777 (0x2d78 bytes) 2024-09-14T07:08:11.583 INFO:teuthology.orchestra.run.smithi145.stdout:651 read 0xd17200 thru 0xd213ac (0xa1ad bytes) 2024-09-14T07:08:11.586 INFO:teuthology.orchestra.run.smithi145.stdout:652 read 0x5ad5400 thru 0x5add1ca (0x7dcb bytes) 2024-09-14T07:08:11.586 INFO:teuthology.orchestra.run.smithi145.stdout:654 read 0x7018200 thru 0x701da57 (0x5858 bytes) 2024-09-14T07:08:11.598 INFO:teuthology.orchestra.run.smithi145.stdout:657 trunc from 0xdc6e000 to 0x7dd3600 2024-09-14T07:08:11.619 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:11.618+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:11.666 INFO:teuthology.orchestra.run.smithi145.stdout:660 trunc from 0x7dd3600 to 0x3cb0400 2024-09-14T07:08:11.684 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:11.685 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:11.685 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:11.685 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:11.685 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:11.700 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:11.721 INFO:teuthology.orchestra.run.smithi145.stdout:661 write 0xd4fe00 thru 0xd5dce2 (0xdee3 bytes) 2024-09-14T07:08:11.724 INFO:teuthology.orchestra.run.smithi145.stdout:662 punch from 0x2ca5600 to 0x2ca9000, (0x3a00 bytes) 2024-09-14T07:08:11.728 INFO:teuthology.orchestra.run.smithi145.stdout:663 read 0x23bfe00 thru 0x23c508b (0x528c bytes) 2024-09-14T07:08:11.728 INFO:teuthology.orchestra.run.smithi145.stdout:666 read 0x457c00 thru 0x45ead9 (0x6eda bytes) 2024-09-14T07:08:11.730 INFO:teuthology.orchestra.run.smithi145.stdout:668 read 0x36a8400 thru 0x36ae150 (0x5d51 bytes) 2024-09-14T07:08:11.747 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:11.746+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:11.768 INFO:teuthology.orchestra.run.smithi145.stdout:670 write 0x82dac00 thru 0x82e18fb (0x6cfc bytes) 2024-09-14T07:08:11.770 INFO:teuthology.orchestra.run.smithi145.stdout:671 write 0x6ce1c00 thru 0x6ce52c4 (0x36c5 bytes) 2024-09-14T07:08:11.770 INFO:teuthology.orchestra.run.smithi145.stdout:673 read 0x3ec7200 thru 0x3eca7c3 (0x35c4 bytes) 2024-09-14T07:08:11.771 INFO:teuthology.orchestra.run.smithi145.stdout:675 read 0x56f9600 thru 0x57085ab (0xefac bytes) 2024-09-14T07:08:11.771 INFO:teuthology.orchestra.run.smithi145.stdout:677 write 0x18f3600 thru 0x18f4e53 (0x1854 bytes) 2024-09-14T07:08:11.771 INFO:teuthology.orchestra.run.smithi145.stdout:681 punch from 0x392f000 to 0x3932c00, (0x3c00 bytes) 2024-09-14T07:08:11.773 INFO:teuthology.orchestra.run.smithi145.stdout:682 read 0x2599a00 thru 0x25a831c (0xe91d bytes) 2024-09-14T07:08:11.807 INFO:teuthology.orchestra.run.smithi145.stdout:684 write 0xc01ea00 thru 0xc029224 (0xa825 bytes) 2024-09-14T07:08:11.821 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:11.818+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:11.833 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:11.833 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:11.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:11.833 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.000983095999999, 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.834 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:11.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:11.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.000208747999999, 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:11.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:11.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:11.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:11.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:795 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.697210+0000", 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.134310159, 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027608120999999999, 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 795 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697210+0000", 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697210+0000", 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697211+0000", 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1769999999999999e-06 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697217+0000", 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9070000000000004e-06 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697236+0000", 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8946000000000001e-05 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697256+0000", 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.723638+0000", 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026382093999999998 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.723761+0000", 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123745 2024-09-14T07:08:11.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.724818+0000", 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010567720000000001 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:796 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 475136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.697278+0000", 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1342418950000002, 2024-09-14T07:08:11.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028063490999999999, 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 796 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697278+0000", 2024-09-14T07:08:11.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697278+0000", 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697280+0000", 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4899999999999999e-06 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697286+0000", 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0859999999999998e-06 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697304+0000", 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7972999999999999e-05 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697317+0000", 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3482999999999999e-05 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.724430+0000", 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027112988000000001 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.724689+0000", 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000258204 2024-09-14T07:08:11.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.725342+0000", 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00065326700000000002 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:802 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.697524+0000", 2024-09-14T07:08:11.853 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1339964870000001, 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028604332999999999, 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 802 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697524+0000", 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697524+0000", 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697526+0000", 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697535+0000", 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.183e-06 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697587+0000", 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2126000000000002e-05 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697609+0000", 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.177e-05 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.725397+0000", 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.027787861000000001 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.725547+0000", 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015062699999999999 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.726128+0000", 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00058068300000000002 2024-09-14T07:08:11.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:804 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.697676+0000", 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1338437219999999, 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029329069999999999, 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 804 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697676+0000", 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697676+0000", 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697684+0000", 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8469999999999996e-06 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697693+0000", 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2230000000000007e-06 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697715+0000", 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.243e-05 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697734+0000", 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8589000000000001e-05 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.726233+0000", 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028499071000000001 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.726387+0000", 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015467200000000001 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.727006+0000", 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061823800000000001 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:806 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.697758+0000", 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.133761706, 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034180242, 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 806 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697758+0000", 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697758+0000", 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697760+0000", 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4929999999999999e-06 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697766+0000", 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697785+0000", 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8859999999999999e-05 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697799+0000", 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4214e-05 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.727105+0000", 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029306226000000001 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.727272+0000", 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016690100000000001 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.731939+0000", 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046663879999999996 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:812 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 483328~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.697848+0000", 2024-09-14T07:08:11.866 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1336724999999999, 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034784387999999999, 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 812 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697848+0000", 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697848+0000", 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697849+0000", 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.48e-06 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697856+0000", 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697876+0000", 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9660000000000002e-05 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.697890+0000", 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.451e-05 2024-09-14T07:08:11.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.732027+0000", 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034136484000000002 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.732162+0000", 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013522299999999999 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.732632+0000", 2024-09-14T07:08:11.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000470118 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:822 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 196608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.699302+0000", 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1322178100000002, 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034320215000000001, 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 822 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699302+0000", 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699302+0000", 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699304+0000", 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7859999999999999e-06 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699313+0000", 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5539999999999998e-06 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699336+0000", 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3342000000000001e-05 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699353+0000", 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6529e-05 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.732715+0000", 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.033362725000000003 2024-09-14T07:08:11.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.732885+0000", 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016930600000000001 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.733623+0000", 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00073797299999999995 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.875 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:824 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 294912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.699393+0000", 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1321268849999999, 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034893485000000002, 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 824 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699393+0000", 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699393+0000", 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699395+0000", 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.64e-06 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699402+0000", 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8709999999999999e-06 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699418+0000", 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6362999999999999e-05 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699432+0000", 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3570999999999999e-05 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.733669+0000", 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034236963000000002 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.733871+0000", 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000202067 2024-09-14T07:08:11.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.734287+0000", 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041601000000000002 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:826 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 491520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.699512+0000", 2024-09-14T07:08:11.880 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1320082149999999, 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035531358999999998, 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 826 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699512+0000", 2024-09-14T07:08:11.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699512+0000", 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699514+0000", 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.283e-06 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699522+0000", 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2349999999999999e-06 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699559+0000", 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6470999999999997e-05 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.699576+0000", 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.734345+0000", 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034768670000000002 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.734554+0000", 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020926699999999999 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.735043+0000", 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048901299999999995 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:840 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.700258+0000", 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1312619819999998, 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035435635, 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.885 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 840 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700258+0000", 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700258+0000", 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700260+0000", 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1440000000000001e-06 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700269+0000", 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1200000000000008e-06 2024-09-14T07:08:11.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700293+0000", 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3989999999999999e-05 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700310+0000", 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6823000000000001e-05 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.735126+0000", 2024-09-14T07:08:11.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.034815617 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.735259+0000", 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133249 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.735694+0000", 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043469199999999998 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.889 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:842 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.700371+0000", 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1311492830000001, 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.03632867, 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 842 2024-09-14T07:08:11.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700371+0000", 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700371+0000", 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700373+0000", 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700381+0000", 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700403+0000", 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2537000000000001e-05 2024-09-14T07:08:11.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700420+0000", 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6460000000000002e-05 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.735767+0000", 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035346900000000001 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.735957+0000", 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019065400000000001 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.736700+0000", 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00074220199999999997 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:844 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 499712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.700451+0000", 2024-09-14T07:08:11.894 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.131068859, 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036920026000000002, 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 844 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700451+0000", 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700451+0000", 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700453+0000", 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5829999999999999e-06 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700459+0000", 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2369999999999997e-06 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700478+0000", 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8842999999999999e-05 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.700492+0000", 2024-09-14T07:08:11.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4326e-05 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.736744+0000", 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036251685999999998 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.736889+0000", 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014479200000000001 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.737371+0000", 2024-09-14T07:08:11.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048255900000000003 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:856 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 212992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.702266+0000", 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1292544790000001, 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035825306000000001, 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 856 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702266+0000", 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702266+0000", 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702268+0000", 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3800000000000001e-06 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702276+0000", 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6320000000000008e-06 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702300+0000", 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4644999999999999e-05 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702330+0000", 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9186000000000001e-05 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.737424+0000", 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035094727999999999 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.737627+0000", 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020246700000000001 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.738091+0000", 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046426799999999999 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:861 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.702379+0000", 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1291414, 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036438989999999997, 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.903 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 861 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702379+0000", 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702379+0000", 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702381+0000", 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.995e-06 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702388+0000", 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1400000000000002e-06 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702406+0000", 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8533999999999999e-05 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702420+0000", 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3585e-05 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.738207+0000", 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035786694000000001 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.738332+0000", 2024-09-14T07:08:11.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012490900000000001 2024-09-14T07:08:11.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.738818+0000", 2024-09-14T07:08:11.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048613299999999998 2024-09-14T07:08:11.907 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:864 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 507904~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.702911+0000", 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1286093849999999, 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036512476000000002, 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 864 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702911+0000", 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702911+0000", 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702913+0000", 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3999999999999999e-06 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.702925+0000", 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2019e-05 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.703069+0000", 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014402600000000001 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.703161+0000", 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1379999999999996e-05 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.738871+0000", 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035710194000000001 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.738999+0000", 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012819 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.739423+0000", 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042426699999999998 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:891 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.708522+0000", 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1229983419999998, 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031514091000000001, 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 891 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.708522+0000", 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.708522+0000", 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.708524+0000", 2024-09-14T07:08:11.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0030000000000001e-06 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.708534+0000", 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8120000000000002e-06 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.708562+0000", 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8776e-05 2024-09-14T07:08:11.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.708926+0000", 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036364500000000001 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.739469+0000", 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030543417999999999 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.739574+0000", 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104045 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.740036+0000", 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000462392 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:894 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.709065+0000", 2024-09-14T07:08:11.917 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1224554520000001, 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031628422000000003, 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 894 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709065+0000", 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709065+0000", 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709066+0000", 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2550000000000001e-06 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709071+0000", 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.147e-06 2024-09-14T07:08:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709089+0000", 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7793000000000001e-05 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709100+0000", 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1474000000000001e-05 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.740087+0000", 2024-09-14T07:08:11.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030986623000000001 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.740212+0000", 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012448200000000001 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.740693+0000", 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048164799999999999 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4558.0:896 3.e 3:772aa6ab:::rbd_data.116148d60084.000000000000000c:6 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:08.709117+0000", 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1224030210000002, 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.032502201000000001, 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4558", 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 896 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.922 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709117+0000", 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709117+0000", 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709118+0000", 2024-09-14T07:08:11.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.124e-06 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709123+0000", 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7639999999999996e-06 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709140+0000", 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6501999999999999e-05 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:11.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.709151+0000", 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1355999999999999e-05 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.740763+0000", 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031612522999999997 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.741182+0000", 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041840399999999999 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:11.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:08.741619+0000", 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043752800000000002 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:11.926 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:11.928 INFO:teuthology.orchestra.run.smithi145.stdout:685 write 0xd9b8600 thru 0xd9c5dd0 (0xd7d1 bytes) 2024-09-14T07:08:11.928 INFO:teuthology.orchestra.run.smithi145.stdout:687 punch from 0x17ad600 to 0x17bc600, (0xf000 bytes) 2024-09-14T07:08:11.928 INFO:teuthology.orchestra.run.smithi145.stdout:688 write 0x91a8800 thru 0x91aaa8f (0x2290 bytes) 2024-09-14T07:08:11.928 INFO:teuthology.orchestra.run.smithi145.stdout:690 write 0x234bc00 thru 0x234febd (0x42be bytes) 2024-09-14T07:08:11.928 INFO:teuthology.orchestra.run.smithi145.stdout:691 trunc from 0xd9c5e00 to 0xbe02200 2024-09-14T07:08:11.928 INFO:teuthology.orchestra.run.smithi145.stdout:693 write 0xcb8200 thru 0xcc5edb (0xdcdc bytes) 2024-09-14T07:08:11.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:11.929 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:11.926+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:11.929 INFO:teuthology.orchestra.run.smithi145.stdout:694 trunc from 0xbe02200 to 0xb61600 2024-09-14T07:08:11.944 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:11.944 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:11.944 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T07:08:11.944 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:08:11.944 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:08:11.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T07:08:12.014 INFO:teuthology.orchestra.run.smithi145.stdout:696 punch from 0x478600 to 0x479a00, (0x1400 bytes) 2024-09-14T07:08:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:697 read 0x426000 thru 0x42c98d (0x698e bytes) 2024-09-14T07:08:12.020 INFO:teuthology.orchestra.run.smithi145.stdout:698 punch from 0x254600 to 0x261200, (0xcc00 bytes) 2024-09-14T07:08:12.022 INFO:teuthology.orchestra.run.smithi145.stdout:702 read 0x2e3c00 thru 0x2e3c0c (0xd bytes) 2024-09-14T07:08:12.028 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:12.026+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:12.045 INFO:teuthology.orchestra.run.smithi145.stdout:704 trunc from 0xb61600 to 0xabd9800 2024-09-14T07:08:12.060 INFO:teuthology.orchestra.run.smithi145.stdout:705 write 0x620ca00 thru 0x620f27e (0x287f bytes) 2024-09-14T07:08:12.062 INFO:teuthology.orchestra.run.smithi145.stdout:706 read 0x6913000 thru 0x69202e1 (0xd2e2 bytes) 2024-09-14T07:08:12.062 INFO:teuthology.orchestra.run.smithi145.stdout:707 read 0x654b400 thru 0x6556b59 (0xb75a bytes) 2024-09-14T07:08:12.063 INFO:teuthology.orchestra.run.smithi145.stdout:709 write 0x3eda000 thru 0x3ee6281 (0xc282 bytes) 2024-09-14T07:08:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:12.093 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:12.093 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:12.093 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:12.101 INFO:teuthology.orchestra.run.smithi145.stdout:712 write 0xe74a000 thru 0xe75196d (0x796e bytes) 2024-09-14T07:08:12.104 INFO:teuthology.orchestra.run.smithi145.stdout:716 write 0x2fa00 thru 0x38a54 (0x9055 bytes) 2024-09-14T07:08:12.104 INFO:teuthology.orchestra.run.smithi145.stdout:717 read 0xbe4200 thru 0xbecdac (0x8bad bytes) 2024-09-14T07:08:12.105 INFO:teuthology.orchestra.run.smithi145.stdout:718 punch from 0x7d3a600 to 0x7d42400, (0x7e00 bytes) 2024-09-14T07:08:12.107 INFO:teuthology.orchestra.run.smithi145.stdout:719 punch from 0xb196000 to 0xb196e00, (0xe00 bytes) 2024-09-14T07:08:12.107 INFO:teuthology.orchestra.run.smithi145.stdout:720 read 0x16ebe00 thru 0x16f9899 (0xda9a bytes) 2024-09-14T07:08:12.107 INFO:teuthology.orchestra.run.smithi145.stdout:721 write 0xd303e00 thru 0xd3063ca (0x25cb bytes) 2024-09-14T07:08:12.108 INFO:teuthology.orchestra.run.smithi145.stdout:722 punch from 0x1fbb000 to 0x1fc3600, (0x8600 bytes) 2024-09-14T07:08:12.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:12.128 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:12.126+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:12.133 INFO:teuthology.orchestra.run.smithi145.stdout:724 trunc from 0xe751a00 to 0xf0e800 2024-09-14T07:08:12.136 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T07:08:12.137 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:12.166 INFO:teuthology.orchestra.run.smithi145.stdout:725 read 0x768800 thru 0x775677 (0xce78 bytes) 2024-09-14T07:08:12.173 INFO:teuthology.orchestra.run.smithi145.stdout:728 trunc from 0xf0e800 to 0x293e000 2024-09-14T07:08:12.217 INFO:teuthology.orchestra.run.smithi145.stdout:729 write 0xe434e00 thru 0xe43fba0 (0xada1 bytes) 2024-09-14T07:08:12.219 INFO:teuthology.orchestra.run.smithi145.stdout:730 read 0x219b000 thru 0x219c029 (0x102a bytes) 2024-09-14T07:08:12.219 INFO:teuthology.orchestra.run.smithi145.stdout:740 punch from 0xa7e0400 to 0xa7efa00, (0xf600 bytes) 2024-09-14T07:08:12.220 INFO:teuthology.orchestra.run.smithi145.stdout:743 write 0x8aae200 thru 0x8ab45ee (0x63ef bytes) 2024-09-14T07:08:12.220 INFO:teuthology.orchestra.run.smithi145.stdout:745 read 0x7846800 thru 0x784dc81 (0x7482 bytes) 2024-09-14T07:08:12.221 INFO:teuthology.orchestra.run.smithi145.stdout:748 write 0x9221000 thru 0x922f41a (0xe41b bytes) 2024-09-14T07:08:12.221 INFO:teuthology.orchestra.run.smithi145.stdout:749 read 0x46d8e00 thru 0x46e1dc8 (0x8fc9 bytes) 2024-09-14T07:08:12.221 INFO:teuthology.orchestra.run.smithi145.stdout:751 write 0x71f4c00 thru 0x7200d75 (0xc176 bytes) 2024-09-14T07:08:12.221 INFO:teuthology.orchestra.run.smithi145.stdout:754 read 0xc925800 thru 0xc92f8fb (0xa0fc bytes) 2024-09-14T07:08:12.221 INFO:teuthology.orchestra.run.smithi145.stdout:757 read 0x28d3800 thru 0x28dbf08 (0x8709 bytes) 2024-09-14T07:08:12.229 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:12.226+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:12.237 INFO:teuthology.orchestra.run.smithi145.stdout:762 trunc from 0xe43fc00 to 0x2ea9800 2024-09-14T07:08:12.280 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:12.281 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:12.281 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:12.281 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:12.281 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:12.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:12.328 INFO:teuthology.orchestra.run.smithi145.stdout:763 write 0xdf04400 thru 0xdf0f5fe (0xb1ff bytes) 2024-09-14T07:08:12.330 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:12.326+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:12.330 INFO:teuthology.orchestra.run.smithi145.stdout:764 write 0xd899400 thru 0xd89b494 (0x2095 bytes) 2024-09-14T07:08:12.342 INFO:teuthology.orchestra.run.smithi145.stdout:765 trunc from 0xdf0f600 to 0x8667600 2024-09-14T07:08:12.377 INFO:teuthology.orchestra.run.smithi145.stdout:766 read 0x4fbe00 thru 0x50797f (0xbb80 bytes) 2024-09-14T07:08:12.384 INFO:teuthology.orchestra.run.smithi145.stdout:767 trunc from 0x8667600 to 0x6891a00 2024-09-14T07:08:12.423 INFO:teuthology.orchestra.run.smithi145.stdout:768 write 0xa6b8200 thru 0xa6c1bff (0x9a00 bytes) 2024-09-14T07:08:12.423 INFO:teuthology.orchestra.run.smithi145.stdout:770 punch from 0x8e3ee00 to 0x8e40c00, (0x1e00 bytes) 2024-09-14T07:08:12.426 INFO:teuthology.orchestra.run.smithi145.stdout:773 punch from 0x6ba3c00 to 0x6ba7600, (0x3a00 bytes) 2024-09-14T07:08:12.426 INFO:teuthology.orchestra.run.smithi145.stdout:775 read 0x32c2a00 thru 0x32c5672 (0x2c73 bytes) 2024-09-14T07:08:12.426 INFO:teuthology.orchestra.run.smithi145.stdout:776 read 0x1d60400 thru 0x1d642d3 (0x3ed4 bytes) 2024-09-14T07:08:12.426 INFO:teuthology.orchestra.run.smithi145.stdout:778 write 0x8171000 thru 0x817ba21 (0xaa22 bytes) 2024-09-14T07:08:12.426 INFO:teuthology.orchestra.run.smithi145.stdout:779 read 0x543be00 thru 0x5447be7 (0xbde8 bytes) 2024-09-14T07:08:12.431 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:12.430+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:12.438 INFO:teuthology.orchestra.run.smithi145.stdout:780 trunc from 0xa6c1c00 to 0x349c200 2024-09-14T07:08:12.475 INFO:teuthology.orchestra.run.smithi145.stdout:783 punch from 0x2341600 to 0x234d000, (0xba00 bytes) 2024-09-14T07:08:12.477 INFO:teuthology.orchestra.run.smithi145.stdout:785 read 0x745200 thru 0x74bce3 (0x6ae4 bytes) 2024-09-14T07:08:12.479 INFO:teuthology.orchestra.run.smithi145.stdout:787 write 0x14c7a00 thru 0x14d25e0 (0xabe1 bytes) 2024-09-14T07:08:12.479 INFO:teuthology.orchestra.run.smithi145.stdout:789 read 0xaeca00 thru 0xaf7bb5 (0xb1b6 bytes) 2024-09-14T07:08:12.480 INFO:teuthology.orchestra.run.smithi145.stdout:791 write 0x1be2800 thru 0x1be5134 (0x2935 bytes) 2024-09-14T07:08:12.489 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:12.489 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:12.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4406.0:1153 3.18 e26/25 3:1f883cd2:::rbd_data.11611b5ca054.000000000000001c:head v 26'3, mlcod=0'0)", 2024-09-14T07:08:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:08:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.461995511000001, 2024-09-14T07:08:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012967347000000001, 2024-09-14T07:08:12.490 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:08:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025605+0000", 2024-09-14T07:08:12.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025607+0000", 2024-09-14T07:08:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0169999999999999e-06 2024-09-14T07:08:12.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025708+0000", 2024-09-14T07:08:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010069900000000001 2024-09-14T07:08:12.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025739+0000", 2024-09-14T07:08:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1086e-05 2024-09-14T07:08:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025759+0000", 2024-09-14T07:08:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-05 2024-09-14T07:08:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025880+0000", 2024-09-14T07:08:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120623 2024-09-14T07:08:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.025918+0000", 2024-09-14T07:08:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8206e-05 2024-09-14T07:08:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038441+0000", 2024-09-14T07:08:12.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012523117 2024-09-14T07:08:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.038573+0000", 2024-09-14T07:08:12.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013165900000000001 2024-09-14T07:08:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.500 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:368 3.1f 3:f8928d1a:::rbd_data.1161a9b4f57.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.4234365740000001, 2024-09-14T07:08:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01725641, 2024-09-14T07:08:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:12.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:12.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:12.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 368 2024-09-14T07:08:12.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.503 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:12.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:12.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064165+0000", 2024-09-14T07:08:12.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.540000000000001e-07 2024-09-14T07:08:12.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064170+0000", 2024-09-14T07:08:12.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8940000000000001e-06 2024-09-14T07:08:12.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064185+0000", 2024-09-14T07:08:12.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5182e-05 2024-09-14T07:08:12.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064193+0000", 2024-09-14T07:08:12.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7999999999999999e-06 2024-09-14T07:08:12.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064281+0000", 2024-09-14T07:08:12.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7517999999999998e-05 2024-09-14T07:08:12.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064365+0000", 2024-09-14T07:08:12.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4728000000000006e-05 2024-09-14T07:08:12.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:12.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064567+0000", 2024-09-14T07:08:12.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201757 2024-09-14T07:08:12.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:12.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068739+0000", 2024-09-14T07:08:12.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041723519999999998 2024-09-14T07:08:12.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:12.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081335+0000", 2024-09-14T07:08:12.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012595492999999999 2024-09-14T07:08:12.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:12.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081366+0000", 2024-09-14T07:08:12.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1538999999999997e-05 2024-09-14T07:08:12.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081421+0000", 2024-09-14T07:08:12.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4193000000000001e-05 2024-09-14T07:08:12.514 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.514 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.515 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.515 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:373 3.1f 3:fcd26671:::rbd_data.1161a9b4f57.0000000000000045:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:12.515 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:12.515 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.4230838349999999, 2024-09-14T07:08:12.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017078697, 2024-09-14T07:08:12.516 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:12.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:12.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:12.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 373 2024-09-14T07:08:12.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.517 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:12.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:12.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064518+0000", 2024-09-14T07:08:12.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:08:12.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064524+0000", 2024-09-14T07:08:12.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9839999999999998e-06 2024-09-14T07:08:12.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064536+0000", 2024-09-14T07:08:12.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2089999999999999e-05 2024-09-14T07:08:12.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064544+0000", 2024-09-14T07:08:12.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1589999999999993e-06 2024-09-14T07:08:12.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065143+0000", 2024-09-14T07:08:12.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059867500000000001 2024-09-14T07:08:12.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065216+0000", 2024-09-14T07:08:12.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3497000000000005e-05 2024-09-14T07:08:12.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:12.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065416+0000", 2024-09-14T07:08:12.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019953899999999999 2024-09-14T07:08:12.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:12.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068744+0000", 2024-09-14T07:08:12.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033285459999999999 2024-09-14T07:08:12.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:12.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081472+0000", 2024-09-14T07:08:12.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012727815 2024-09-14T07:08:12.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:12.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081528+0000", 2024-09-14T07:08:12.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5958000000000002e-05 2024-09-14T07:08:12.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081596+0000", 2024-09-14T07:08:12.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7489000000000004e-05 2024-09-14T07:08:12.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.529 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:386 3.1f 3:fbac3711:::rbd_data.1161a9b4f57.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:12.529 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:12.530 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.4213399920000001, 2024-09-14T07:08:12.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015471182999999999, 2024-09-14T07:08:12.530 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:12.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:12.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:12.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 386 2024-09-14T07:08:12.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.531 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:12.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:12.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066262+0000", 2024-09-14T07:08:12.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5600000000000004e-07 2024-09-14T07:08:12.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066268+0000", 2024-09-14T07:08:12.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0809999999999999e-06 2024-09-14T07:08:12.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066279+0000", 2024-09-14T07:08:12.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1389999999999999e-05 2024-09-14T07:08:12.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066286+0000", 2024-09-14T07:08:12.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8959999999999997e-06 2024-09-14T07:08:12.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066393+0000", 2024-09-14T07:08:12.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107413 2024-09-14T07:08:12.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066451+0000", 2024-09-14T07:08:12.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7454000000000003e-05 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066556+0000", 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105326 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:12.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068748+0000", 2024-09-14T07:08:12.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002192285 2024-09-14T07:08:12.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:12.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081641+0000", 2024-09-14T07:08:12.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012892280000000001 2024-09-14T07:08:12.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:12.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081688+0000", 2024-09-14T07:08:12.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-09-14T07:08:12.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081732+0000", 2024-09-14T07:08:12.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4132999999999999e-05 2024-09-14T07:08:12.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.543 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.543 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:389 3.1f 3:fddcb068:::rbd_data.1161a9b4f57.0000000000000060:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:12.543 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.4210941760000004, 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015361717, 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 389 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1200000000000002e-07 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066518+0000", 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0913999999999999e-05 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066528+0000", 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066533+0000", 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9910000000000002e-06 2024-09-14T07:08:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066729+0000", 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019615400000000001 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066810+0000", 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1149999999999994e-05 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.067039+0000", 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022877600000000001 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068752+0000", 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001713542 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:12.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081789+0000", 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013036384999999999 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081828+0000", 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8772000000000001e-05 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081868+0000", 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0762e-05 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.549 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:392 3.12 3:4c441206:::rbd_data.1161a9b4f57.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.420956511, 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014904545, 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:12.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 392 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066645+0000", 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066647+0000", 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6079999999999998e-06 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066653+0000", 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.631e-06 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066667+0000", 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4051e-05 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066706+0000", 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8621999999999997e-05 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066767+0000", 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1723000000000005e-05 2024-09-14T07:08:12.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066894+0000", 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012675300000000001 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068486+0000", 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001591692 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081434+0000", 2024-09-14T07:08:12.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012948337000000001 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081490+0000", 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5551999999999999e-05 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081549+0000", 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9114999999999999e-05 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:281 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 499712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.467851+0000", 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.019749923, 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012936283999999999, 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 281 2024-09-14T07:08:12.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467851+0000", 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467851+0000", 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467852+0000", 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1039999999999999e-06 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467857+0000", 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4960000000000004e-06 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467875+0000", 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7917000000000001e-05 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467889+0000", 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3322999999999999e-05 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.480309+0000", 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012420168000000001 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.480563+0000", 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025367400000000001 2024-09-14T07:08:12.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.480787+0000", 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022460199999999999 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:282 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.467908+0000", 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0196931569999999, 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013537126, 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 282 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467908+0000", 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467908+0000", 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467909+0000", 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.043e-06 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467914+0000", 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3739999999999997e-06 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467931+0000", 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7333000000000002e-05 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467945+0000", 2024-09-14T07:08:12.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3540000000000001e-05 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.480914+0000", 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012968706999999999 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481156+0000", 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242814 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481445+0000", 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028831499999999998 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:283 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.467964+0000", 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0196364999999998, 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014115351, 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 283 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467964+0000", 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467964+0000", 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467965+0000", 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2160000000000001e-06 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467971+0000", 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7139999999999998e-06 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467989+0000", 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7926000000000001e-05 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468003+0000", 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3689999999999999e-05 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481562+0000", 2024-09-14T07:08:12.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558849 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481785+0000", 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000222888 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482080+0000", 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000295068 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:284 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468066+0000", 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0195350950000002, 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014560095, 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 284 2024-09-14T07:08:12.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468066+0000", 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468066+0000", 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468067+0000", 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.547e-06 2024-09-14T07:08:12.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468075+0000", 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.593e-06 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468097+0000", 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2226999999999999e-05 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468112+0000", 2024-09-14T07:08:12.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5077e-05 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482192+0000", 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014080004 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482415+0000", 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022281499999999999 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482626+0000", 2024-09-14T07:08:12.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021083199999999999 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:285 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 344064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468162+0000", 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0194390389999999, 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014852038999999999, 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 285 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468162+0000", 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468162+0000", 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468163+0000", 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.517e-06 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468175+0000", 2024-09-14T07:08:12.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468197+0000", 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1610000000000001e-05 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468212+0000", 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4724e-05 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482731+0000", 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014519148000000001 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482864+0000", 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013291900000000001 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483014+0000", 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015000500000000001 2024-09-14T07:08:12.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:286 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0193646520000001, 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01542496, 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 286 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:12.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468238+0000", 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3960000000000001e-06 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468244+0000", 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468267+0000", 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3603000000000001e-05 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468278+0000", 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0883000000000001e-05 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483169+0000", 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014890970999999999 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483418+0000", 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024896800000000002 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483661+0000", 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024296499999999999 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:287 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.583 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0193076799999998, 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015987674, 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 287 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468294+0000", 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6300000000000004e-07 2024-09-14T07:08:12.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468329+0000", 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4799999999999999e-05 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468349+0000", 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0299999999999999e-05 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468363+0000", 2024-09-14T07:08:12.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4284000000000001e-05 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483821+0000", 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015458120000000001 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484068+0000", 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000246281 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484281+0000", 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000213026 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:288 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0192129799999998, 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016408030000000001, 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 288 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:12.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468389+0000", 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.64e-06 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468396+0000", 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468411+0000", 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5153e-05 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468419+0000", 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7039999999999994e-06 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484366+0000", 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015946689999999999 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484617+0000", 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025165400000000003 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484796+0000", 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017846299999999999 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.592 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:289 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 122880~6656] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0191690009999999, 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111327999999999, 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 289 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468433+0000", 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1999999999999999e-07 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468436+0000", 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3969999999999998e-06 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468445+0000", 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9800000000000004e-06 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468455+0000", 2024-09-14T07:08:12.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6900000000000004e-06 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484948+0000", 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016493799 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485216+0000", 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026811499999999998 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485543+0000", 2024-09-14T07:08:12.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032662700000000002 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:290 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0191359540000002, 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017722122999999999, 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 290 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4000000000000001e-07 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468469+0000", 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7529999999999999e-06 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468478+0000", 2024-09-14T07:08:12.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6799999999999999e-06 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468485+0000", 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.153e-06 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485735+0000", 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017249970999999999 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485956+0000", 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022074799999999999 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486187+0000", 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023117799999999999 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:291 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:12.601 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0191048780000003, 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018193804000000001, 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 291 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:12.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468497+0000", 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468502+0000", 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5439999999999998e-06 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468531+0000", 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8898999999999999e-05 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468543+0000", 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1756999999999999e-05 2024-09-14T07:08:12.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486295+0000", 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017751423999999998 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486492+0000", 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019711899999999999 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486690+0000", 2024-09-14T07:08:12.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019808099999999999 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:292 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0189980749999998, 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018553717000000001, 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 292 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468604+0000", 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5999999999999999e-06 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468611+0000", 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7800000000000003e-06 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468633+0000", 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1710000000000001e-05 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468647+0000", 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4209999999999999e-05 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486764+0000", 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018116538000000001 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486948+0000", 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018412999999999999 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487156+0000", 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020874899999999999 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:293 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.0189315130000001, 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019096847, 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 293 2024-09-14T07:08:12.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468671+0000", 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468676+0000", 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.857e-06 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468694+0000", 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7629999999999999e-05 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468712+0000", 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7762999999999999e-05 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487288+0000", 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018576294 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487555+0000", 2024-09-14T07:08:12.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026692399999999998 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487766+0000", 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021110899999999999 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4579.0:2117 3.17 3:ec6f4503:::rbd_data.11616dafc67f.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write 294912~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:12.245915+0000", 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.24168531200000001, 2024-09-14T07:08:12.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013416216999999999, 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4579", 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2117 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245915+0000", 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245915+0000", 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245917+0000", 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0190000000000001e-06 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245977+0000", 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9274000000000002e-05 2024-09-14T07:08:12.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245996+0000", 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9734000000000001e-05 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246006+0000", 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3260000000000002e-06 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246073+0000", 2024-09-14T07:08:12.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7402000000000003e-05 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246199+0000", 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000125401 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246311+0000", 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011273900000000001 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:12.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.249360+0000", 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030491060000000002 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.259275+0000", 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0099142669999999992 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.259308+0000", 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3117000000000002e-05 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:12.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.259332+0000", 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3832000000000002e-05 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:12.621 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:12.622 INFO:teuthology.orchestra.run.smithi145.stdout:794 write 0xd619600 thru 0xd62545c (0xbe5d bytes) 2024-09-14T07:08:12.622 INFO:teuthology.orchestra.run.smithi145.stdout:799 clone 7 order 24 su 262144 sc 13 2024-09-14T07:08:12.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:12.623 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:12.622+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:12.723 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:12.722+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:12.802 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:12.802 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:12.803 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:12.803 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:12.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:12.863 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:12.862+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:12.926 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:12.922+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:12.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:08:13.011 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:13.012 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:13.026 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:13.022+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:13.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:13.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:13.106 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone6 from 0xd625600 (overlap 0xb61600) to 0x6d4000 2024-09-14T07:08:13.134 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:13.134+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:939 3.2 3:40fc327c:::rbd_data.11611b5ca054.000000000000002e:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3473408~57344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:53.854240+0000", 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.313281979999999, 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012138079, 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 939 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854240+0000", 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854240+0000", 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854243+0000", 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.597e-06 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854526+0000", 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000283038 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854570+0000", 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3726000000000001e-05 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854594+0000", 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3672999999999998e-05 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.854779+0000", 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018488900000000001 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.855023+0000", 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024445800000000002 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.855488+0000", 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00046545199999999999 2024-09-14T07:08:13.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.865325+0000", 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098367479999999993 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866097+0000", 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00077223199999999997 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866258+0000", 2024-09-14T07:08:13.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160927 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:53.866378+0000", 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000119339 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1168 3.2 3:42220d69:::rbd_data.11611b5ca054.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3526656~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.133495240999999, 2024-09-14T07:08:13.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014557588, 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1168 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034029+0000", 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9270000000000001e-06 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034081+0000", 2024-09-14T07:08:13.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1313000000000002e-05 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034115+0000", 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4156000000000003e-05 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034131+0000", 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6350000000000001e-05 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034276+0000", 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014510900000000001 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034400+0000", 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012352899999999999 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034616+0000", 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021601199999999999 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.045899+0000", 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011283519000000001 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048374+0000", 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024744379999999998 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048466+0000", 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1952000000000003e-05 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048584+0000", 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011828300000000001 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1170 3.2 3:40d5ec5b:::rbd_data.11611b5ca054.000000000000002f:head [set-alloc-hint object_size 4194304 write_size 4194304,write 4005888~61440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.034753+0000", 2024-09-14T07:08:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.132768729999999, 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014288159999999999, 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1170 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034753+0000", 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034753+0000", 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034755+0000", 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.857e-06 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034813+0000", 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8686e-05 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034835+0000", 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1916000000000002e-05 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034850+0000", 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4627000000000001e-05 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.037339+0000", 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024895239999999999 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.037555+0000", 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000215575 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.037781+0000", 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022602500000000001 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.046600+0000", 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0088193349999999993 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048869+0000", 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022680930000000001 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048962+0000", 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3721999999999995e-05 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.049041+0000", 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8800000000000004e-05 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:243 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.456924+0000", 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7105979339999999, 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016379797000000001, 2024-09-14T07:08:13.187 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456924+0000", 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456924+0000", 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456925+0000", 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5300000000000002e-07 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456929+0000", 2024-09-14T07:08:13.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9169999999999999e-06 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456940+0000", 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1909999999999999e-05 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456948+0000", 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7530000000000001e-06 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463086+0000", 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0061378700000000001 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463225+0000", 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013878600000000001 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463974+0000", 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00074900399999999995 2024-09-14T07:08:13.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.472880+0000", 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0089065640000000005 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473050+0000", 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016923900000000001 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473194+0000", 2024-09-14T07:08:13.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014389200000000001 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473303+0000", 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010990899999999999 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:244 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.458447+0000", 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7090745949999997, 2024-09-14T07:08:13.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019738037, 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 244 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458447+0000", 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458447+0000", 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458450+0000", 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8030000000000001e-06 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458465+0000", 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5440000000000001e-05 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458504+0000", 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8495999999999999e-05 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458526+0000", 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1857000000000001e-05 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.466150+0000", 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0076242539999999996 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.466381+0000", 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023085099999999999 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467333+0000", 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095189000000000005 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475338+0000", 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080058879999999992 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.477979+0000", 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0026403799999999999 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478110+0000", 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013101599999999999 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478185+0000", 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5161999999999996e-05 2024-09-14T07:08:13.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:245 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.459256+0000", 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7082651250000005, 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019305170999999999, 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 245 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459256+0000", 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459256+0000", 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459259+0000", 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459271+0000", 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-05 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459298+0000", 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6789999999999999e-05 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459316+0000", 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7920000000000001e-05 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468338+0000", 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0090225199999999992 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468608+0000", 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026966799999999998 2024-09-14T07:08:13.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469695+0000", 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010870420000000001 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475354+0000", 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056588259999999996 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478294+0000", 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029405630000000002 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478415+0000", 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121036 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478562+0000", 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014641599999999999 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:246 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.460112+0000", 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7074097679999998, 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018752072000000002, 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 246 2024-09-14T07:08:13.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460112+0000", 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460112+0000", 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460114+0000", 2024-09-14T07:08:13.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3029999999999998e-06 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460128+0000", 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3509999999999999e-05 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460180+0000", 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1910000000000003e-05 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460201+0000", 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1733e-05 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.470938+0000", 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01073671 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.471144+0000", 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020582599999999999 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.471956+0000", 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00081192999999999999 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.477819+0000", 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0058631780000000001 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478701+0000", 2024-09-14T07:08:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00088250899999999996 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478805+0000", 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010378300000000001 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478864+0000", 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8680000000000001e-05 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:247 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.460916+0000", 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7066060209999998, 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018220002999999999, 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 247 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460916+0000", 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460916+0000", 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460918+0000", 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.317e-06 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460933+0000", 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4876e-05 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460963+0000", 2024-09-14T07:08:13.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0069999999999998e-05 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460982+0000", 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9596e-05 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473450+0000", 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012467974 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473685+0000", 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000234572 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.474344+0000", 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00065870800000000004 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478984+0000", 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046402270000000002 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479005+0000", 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0704e-05 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479077+0000", 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2612000000000002e-05 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479136+0000", 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8347000000000002e-05 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:248 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.462023+0000", 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7054983200000002, 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02295055, 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 248 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462023+0000", 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462023+0000", 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462025+0000", 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.827e-06 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462035+0000", 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7599999999999997e-06 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462056+0000", 2024-09-14T07:08:13.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1387e-05 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462069+0000", 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.311e-05 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475441+0000", 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013371686000000001 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475677+0000", 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000235832 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.476589+0000", 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00091255600000000002 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483711+0000", 2024-09-14T07:08:13.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0071217040000000004 2024-09-14T07:08:13.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484699+0000", 2024-09-14T07:08:13.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00098739599999999998 2024-09-14T07:08:13.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484857+0000", 2024-09-14T07:08:13.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015856600000000001 2024-09-14T07:08:13.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484974+0000", 2024-09-14T07:08:13.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011672599999999999 2024-09-14T07:08:13.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.224 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.224 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:249 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.463149+0000", 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7043726850000001, 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022312041000000001, 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 249 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463149+0000", 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463149+0000", 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463151+0000", 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-09-14T07:08:13.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463164+0000", 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3047000000000001e-05 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463197+0000", 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2790000000000003e-05 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463218+0000", 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0639999999999999e-05 2024-09-14T07:08:13.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479221+0000", 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016003632 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479350+0000", 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012889600000000001 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479965+0000", 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00061449499999999995 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485152+0000", 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0051874069999999998 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485204+0000", 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-05 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485351+0000", 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014702199999999999 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485461+0000", 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110429 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:250 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.464199+0000", 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7033230799999997, 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026385663, 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 250 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464199+0000", 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464199+0000", 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464201+0000", 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2730000000000001e-06 2024-09-14T07:08:13.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464214+0000", 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3677e-05 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464249+0000", 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4496999999999999e-05 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464275+0000", 2024-09-14T07:08:13.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5948999999999998e-05 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.480967+0000", 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016691711000000001 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481101+0000", 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000134206 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482030+0000", 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00092936600000000005 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485642+0000", 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036116920000000001 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490383+0000", 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0047415640000000002 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490520+0000", 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000136372 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490584+0000", 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4356000000000005e-05 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:251 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.465333+0000", 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7021883320000004, 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027876171000000002, 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 251 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465333+0000", 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465333+0000", 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465336+0000", 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6129999999999998e-06 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465371+0000", 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.468e-05 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465400+0000", 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9563000000000001e-05 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465420+0000", 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0302999999999999e-05 2024-09-14T07:08:13.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485814+0000", 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020393593000000002 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486067+0000", 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025264100000000001 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487153+0000", 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001085899 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.492983+0000", 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0058306779999999997 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493005+0000", 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1759999999999998e-05 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493092+0000", 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6599000000000002e-05 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493209+0000", 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117842 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:295 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.242 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6979214379999998, 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028784614, 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 295 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469603+0000", 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469619+0000", 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6163000000000001e-05 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469646+0000", 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6040000000000001e-05 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469660+0000", 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4843e-05 2024-09-14T07:08:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488319+0000", 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018658726 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488567+0000", 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024742500000000001 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.489393+0000", 2024-09-14T07:08:13.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082641099999999999 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493423+0000", 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040299050000000003 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498065+0000", 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046422310000000001 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498254+0000", 2024-09-14T07:08:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000189278 2024-09-14T07:08:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498385+0000", 2024-09-14T07:08:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013043899999999999 2024-09-14T07:08:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:296 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.473652+0000", 2024-09-14T07:08:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6938698460000001, 2024-09-14T07:08:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.025354419, 2024-09-14T07:08:13.250 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 296 2024-09-14T07:08:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473652+0000", 2024-09-14T07:08:13.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473652+0000", 2024-09-14T07:08:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473655+0000", 2024-09-14T07:08:13.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7999999999999999e-06 2024-09-14T07:08:13.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473670+0000", 2024-09-14T07:08:13.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5153e-05 2024-09-14T07:08:13.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473736+0000", 2024-09-14T07:08:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6270000000000001e-05 2024-09-14T07:08:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473758+0000", 2024-09-14T07:08:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2405999999999999e-05 2024-09-14T07:08:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490685+0000", 2024-09-14T07:08:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016926566000000001 2024-09-14T07:08:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490826+0000", 2024-09-14T07:08:13.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000140667 2024-09-14T07:08:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.491817+0000", 2024-09-14T07:08:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0009915010000000001 2024-09-14T07:08:13.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498557+0000", 2024-09-14T07:08:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006739798 2024-09-14T07:08:13.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498749+0000", 2024-09-14T07:08:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019212 2024-09-14T07:08:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498885+0000", 2024-09-14T07:08:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013625900000000001 2024-09-14T07:08:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499006+0000", 2024-09-14T07:08:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012087900000000001 2024-09-14T07:08:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:298 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.475423+0000", 2024-09-14T07:08:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6920981030000002, 2024-09-14T07:08:13.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023859328999999999, 2024-09-14T07:08:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 298 2024-09-14T07:08:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475423+0000", 2024-09-14T07:08:13.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475423+0000", 2024-09-14T07:08:13.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475426+0000", 2024-09-14T07:08:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.12e-06 2024-09-14T07:08:13.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475437+0000", 2024-09-14T07:08:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1783000000000001e-05 2024-09-14T07:08:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475463+0000", 2024-09-14T07:08:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5746e-05 2024-09-14T07:08:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475482+0000", 2024-09-14T07:08:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.906e-05 2024-09-14T07:08:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493548+0000", 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018066181000000001 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493806+0000", 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025763499999999998 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.494898+0000", 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001091921 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498716+0000", 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0038182950000000002 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499050+0000", 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033355199999999998 2024-09-14T07:08:13.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499174+0000", 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012412499999999999 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499283+0000", 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108911 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:300 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.478508+0000", 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6890135170000002, 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026193111000000002, 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 300 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478508+0000", 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478508+0000", 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478510+0000", 2024-09-14T07:08:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.317e-06 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478524+0000", 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.366e-05 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478552+0000", 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7835999999999999e-05 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478571+0000", 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9466e-05 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.496068+0000", 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017496989000000001 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.496336+0000", 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026767400000000003 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.497277+0000", 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00094107000000000001 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504404+0000", 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0071268549999999996 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504494+0000", 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0532000000000007e-05 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504602+0000", 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010760999999999999 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504701+0000", 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9102000000000001e-05 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:306 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.280 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.481121+0000", 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6864005640000004, 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023939642000000001, 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 306 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481121+0000", 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481121+0000", 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481124+0000", 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481138+0000", 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4574e-05 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481173+0000", 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4069999999999997e-05 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481197+0000", 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4243000000000002e-05 2024-09-14T07:08:13.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499406+0000", 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018209672999999999 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499598+0000", 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019118800000000001 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.500323+0000", 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000724879 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504768+0000", 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044458279999999998 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504871+0000", 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102222 2024-09-14T07:08:13.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504948+0000", 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7146999999999998e-05 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.505061+0000", 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112942 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.286 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:310 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.484907+0000", 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6826144029999996, 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023074125000000001, 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 310 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484907+0000", 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484907+0000", 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484910+0000", 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484924+0000", 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3923e-05 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484955+0000", 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1309999999999997e-05 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484975+0000", 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9792999999999998e-05 2024-09-14T07:08:13.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.501562+0000", 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016586825999999999 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.501825+0000", 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026327399999999997 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.502909+0000", 2024-09-14T07:08:13.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001083922 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507438+0000", 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045286409999999999 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507572+0000", 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000134139 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507884+0000", 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031209800000000002 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507981+0000", 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7509000000000004e-05 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:315 3.b 3:d73f643a:::rbd_data.1161dbabc1ae.0000000000000008:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.486826+0000", 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6806956280000005, 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017581507999999999, 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 315 2024-09-14T07:08:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486826+0000", 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486826+0000", 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486829+0000", 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6570000000000001e-06 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486841+0000", 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2830000000000001e-05 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486876+0000", 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4585999999999997e-05 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486897+0000", 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.09e-05 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.503971+0000", 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017073748 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504162+0000", 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019167199999999999 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504407+0000", 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024511500000000001 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:316 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.488078+0000", 2024-09-14T07:08:13.298 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6794440909999997, 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021237334, 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 316 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488078+0000", 2024-09-14T07:08:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488078+0000", 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488081+0000", 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3459999999999998e-06 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:13.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488095+0000", 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4273999999999999e-05 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488152+0000", 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6784999999999999e-05 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488173+0000", 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0653999999999999e-05 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.505184+0000", 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017010952999999999 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.505351+0000", 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000167508 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.506477+0000", 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126242 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.508648+0000", 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021702829999999998 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.509219+0000", 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00057135599999999997 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.509274+0000", 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4920999999999998e-05 2024-09-14T07:08:13.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.509315+0000", 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1012e-05 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:13.304 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:13.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:13.306 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:13.306 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:13.307 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:13.307 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:13.307 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:13.308 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:13.308 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:13.302+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:13.326 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:13.326 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:13.347 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:13.347 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:13.407 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:13.406+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:13.444 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:13.445 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:13.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:13.509 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:13.506+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:13.609 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:13.606+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:13.636 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:13.636 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:13.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:13.666 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T07:08:13.709 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:13.706+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:13.789 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:13.789 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:13.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:13.810 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:13.806+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:13.911 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:13.910+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:13.972 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:13.972 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:08:13.987 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:14.035 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:14.034+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:14.113 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:14.110+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:14.178 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:14.179 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:14.197 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:14.239 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:14.238+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:14.315 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:14.314+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:14.385 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:14.402 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:14.443 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:14.442+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:14.516 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:14.514+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:14.578 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:14.578 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:14.592 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:14.617 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:14.614+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:14.718 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:14.718+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:14.774 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:14.774 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:14.790 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:14.835 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:14.834+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:14.920 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:14.918+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:14.939 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:14.939 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:14.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:15.043 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:15.042+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:15.122 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:15.118+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:15.123 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:15.123 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:15.137 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:15.223 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:15.222+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:15.280 INFO:teuthology.orchestra.run.smithi145.stdout:801 read 0x6dac400 thru 0x6db90a6 (0xcca7 bytes) 2024-09-14T07:08:15.281 INFO:teuthology.orchestra.run.smithi145.stdout:803 clone 8 order 24 su 65536 sc 15 2024-09-14T07:08:15.302 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:15.302 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:15.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:15.324 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:15.322+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:15.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:15.425 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:15.422+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:15.437 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:15.438 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:15.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:15.522 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:15.523 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:15.525 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:15.522+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:15.539 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:15.556 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:15.577 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:15.577 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:15.582 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:15.582 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:15.583 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:15.583 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:15.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:15.599 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:15.599 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:15.621 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:15.621 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:15.626 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:15.622+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:15.715 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:15.715 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:15.726 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:15.726+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:15.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:15.828 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:15.826+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:15.899 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:15.899 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:15.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:368 3.1f 3:f8928d1a:::rbd_data.1161a9b4f57.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8333947619999993, 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01725641, 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:15.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 368 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064165+0000", 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.540000000000001e-07 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064170+0000", 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8940000000000001e-06 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064185+0000", 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5182e-05 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064193+0000", 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7999999999999999e-06 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064281+0000", 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7517999999999998e-05 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064365+0000", 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4728000000000006e-05 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064567+0000", 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201757 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068739+0000", 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041723519999999998 2024-09-14T07:08:15.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081335+0000", 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012595492999999999 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081366+0000", 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1538999999999997e-05 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081421+0000", 2024-09-14T07:08:15.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4193000000000001e-05 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:373 3.1f 3:fcd26671:::rbd_data.1161a9b4f57.0000000000000045:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8330420230000009, 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017078697, 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 373 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064518+0000", 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064524+0000", 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9839999999999998e-06 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064536+0000", 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2089999999999999e-05 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064544+0000", 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1589999999999993e-06 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065143+0000", 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059867500000000001 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065216+0000", 2024-09-14T07:08:15.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3497000000000005e-05 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065416+0000", 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019953899999999999 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068744+0000", 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033285459999999999 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081472+0000", 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012727815 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081528+0000", 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5958000000000002e-05 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081596+0000", 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7489000000000004e-05 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:386 3.1f 3:fbac3711:::rbd_data.1161a9b4f57.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8312981799999992, 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015471182999999999, 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 386 2024-09-14T07:08:15.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066262+0000", 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5600000000000004e-07 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066268+0000", 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0809999999999999e-06 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066279+0000", 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1389999999999999e-05 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066286+0000", 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8959999999999997e-06 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066393+0000", 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107413 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066451+0000", 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7454000000000003e-05 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066556+0000", 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105326 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068748+0000", 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002192285 2024-09-14T07:08:15.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081641+0000", 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012892280000000001 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081688+0000", 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081732+0000", 2024-09-14T07:08:15.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4132999999999999e-05 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:389 3.1f 3:fddcb068:::rbd_data.1161a9b4f57.0000000000000060:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8310523639999996, 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015361717, 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 389 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1200000000000002e-07 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066518+0000", 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0913999999999999e-05 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066528+0000", 2024-09-14T07:08:15.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066533+0000", 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9910000000000002e-06 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066729+0000", 2024-09-14T07:08:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019615400000000001 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066810+0000", 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1149999999999994e-05 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.067039+0000", 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022877600000000001 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:15.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068752+0000", 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001713542 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081789+0000", 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013036384999999999 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081828+0000", 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8772000000000001e-05 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081868+0000", 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0762e-05 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:392 3.12 3:4c441206:::rbd_data.1161a9b4f57.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8309146990000009, 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014904545, 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 392 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066645+0000", 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066647+0000", 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6079999999999998e-06 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066653+0000", 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.631e-06 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066667+0000", 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4051e-05 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066706+0000", 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8621999999999997e-05 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066767+0000", 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1723000000000005e-05 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066894+0000", 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012675300000000001 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068486+0000", 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001591692 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081434+0000", 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012948337000000001 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081490+0000", 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5551999999999999e-05 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081549+0000", 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9114999999999999e-05 2024-09-14T07:08:15.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:282 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 16384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.467908+0000", 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4296513449999999, 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013537126, 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 282 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467908+0000", 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467908+0000", 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467909+0000", 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.043e-06 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467914+0000", 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3739999999999997e-06 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467931+0000", 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7333000000000002e-05 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467945+0000", 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3540000000000001e-05 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.480914+0000", 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012968706999999999 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481156+0000", 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242814 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481445+0000", 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028831499999999998 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:283 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.467964+0000", 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4295946879999999, 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014115351, 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 283 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467964+0000", 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467964+0000", 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467965+0000", 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2160000000000001e-06 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467971+0000", 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7139999999999998e-06 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467989+0000", 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7926000000000001e-05 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468003+0000", 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3689999999999999e-05 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481562+0000", 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558849 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481785+0000", 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000222888 2024-09-14T07:08:15.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482080+0000", 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000295068 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:284 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 180224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468066+0000", 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4294932830000002, 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014560095, 2024-09-14T07:08:15.939 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 284 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468066+0000", 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468066+0000", 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468067+0000", 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.547e-06 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468075+0000", 2024-09-14T07:08:15.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.593e-06 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468097+0000", 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2226999999999999e-05 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468112+0000", 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5077e-05 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482192+0000", 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014080004 2024-09-14T07:08:15.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482415+0000", 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022281499999999999 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482626+0000", 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021083199999999999 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.943 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:285 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 344064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468162+0000", 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4293972269999999, 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014852038999999999, 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 285 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468162+0000", 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468162+0000", 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468163+0000", 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.517e-06 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468175+0000", 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468197+0000", 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1610000000000001e-05 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468212+0000", 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4724e-05 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482731+0000", 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014519148000000001 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482864+0000", 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013291900000000001 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483014+0000", 2024-09-14T07:08:15.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015000500000000001 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:286 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4293228400000002, 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01542496, 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 286 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468238+0000", 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3960000000000001e-06 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468244+0000", 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468267+0000", 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3603000000000001e-05 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468278+0000", 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0883000000000001e-05 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483169+0000", 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014890970999999999 2024-09-14T07:08:15.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483418+0000", 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024896800000000002 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483661+0000", 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024296499999999999 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.952 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:287 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4292658679999999, 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015987674, 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 287 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468294+0000", 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6300000000000004e-07 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468329+0000", 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4799999999999999e-05 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468349+0000", 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0299999999999999e-05 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468363+0000", 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4284000000000001e-05 2024-09-14T07:08:15.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483821+0000", 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015458120000000001 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484068+0000", 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000246281 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484281+0000", 2024-09-14T07:08:15.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000213026 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:288 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4291711679999999, 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016408030000000001, 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 288 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468389+0000", 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.64e-06 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468396+0000", 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468411+0000", 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5153e-05 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468419+0000", 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7039999999999994e-06 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484366+0000", 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015946689999999999 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484617+0000", 2024-09-14T07:08:15.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025165400000000003 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484796+0000", 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017846299999999999 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:289 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 122880~6656] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4291271889999999, 2024-09-14T07:08:15.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111327999999999, 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 289 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468433+0000", 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1999999999999999e-07 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468436+0000", 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3969999999999998e-06 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468445+0000", 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9800000000000004e-06 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468455+0000", 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6900000000000004e-06 2024-09-14T07:08:15.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484948+0000", 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016493799 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485216+0000", 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026811499999999998 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485543+0000", 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032662700000000002 2024-09-14T07:08:15.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:290 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4290941420000003, 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017722122999999999, 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 290 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4000000000000001e-07 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468469+0000", 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7529999999999999e-06 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468478+0000", 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6799999999999999e-06 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468485+0000", 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.153e-06 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485735+0000", 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017249970999999999 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485956+0000", 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022074799999999999 2024-09-14T07:08:15.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486187+0000", 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023117799999999999 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:291 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4290630660000003, 2024-09-14T07:08:15.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018193804000000001, 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 291 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:15.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468497+0000", 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468502+0000", 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5439999999999998e-06 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468531+0000", 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8898999999999999e-05 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468543+0000", 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1756999999999999e-05 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486295+0000", 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017751423999999998 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486492+0000", 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019711899999999999 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486690+0000", 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019808099999999999 2024-09-14T07:08:15.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:292 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4289562629999999, 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018553717000000001, 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 292 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468604+0000", 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5999999999999999e-06 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468611+0000", 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7800000000000003e-06 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468633+0000", 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1710000000000001e-05 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468647+0000", 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4209999999999999e-05 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486764+0000", 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018116538000000001 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486948+0000", 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018412999999999999 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487156+0000", 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020874899999999999 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:293 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:15.979 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.4288897010000001, 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019096847, 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 293 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468671+0000", 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468676+0000", 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.857e-06 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468694+0000", 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7629999999999999e-05 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468712+0000", 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7762999999999999e-05 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487288+0000", 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018576294 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487555+0000", 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026692399999999998 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487766+0000", 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021110899999999999 2024-09-14T07:08:15.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4579.0:2117 3.17 3:ec6f4503:::rbd_data.11616dafc67f.00000000000000e0:head [set-alloc-hint object_size 524288 write_size 524288,write 294912~32768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:12.245915+0000", 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.6516435, 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013416216999999999, 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4579", 2024-09-14T07:08:15.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2117 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245915+0000", 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245915+0000", 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245917+0000", 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0190000000000001e-06 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245977+0000", 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9274000000000002e-05 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.245996+0000", 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9734000000000001e-05 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246006+0000", 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3260000000000002e-06 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246073+0000", 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7402000000000003e-05 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246199+0000", 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000125401 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.246311+0000", 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011273900000000001 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.249360+0000", 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030491060000000002 2024-09-14T07:08:15.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.259275+0000", 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0099142669999999992 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.259308+0000", 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3117000000000002e-05 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:12.259332+0000", 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3832000000000002e-05 2024-09-14T07:08:15.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4579.0:3184 3.1b 3:d843d9ee:::rbd_data.11616dafc67f.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:13.187118+0000", 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.7104407589999999, 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013047626, 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4579", 2024-09-14T07:08:15.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3184 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187118+0000", 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187118+0000", 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187119+0000", 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5499999999999998e-07 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187267+0000", 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014825400000000001 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187279+0000", 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1688000000000001e-05 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187291+0000", 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.22e-05 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.190658+0000", 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003367166 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.190721+0000", 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2424999999999999e-05 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.190882+0000", 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016148000000000001 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.193693+0000", 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028113299999999999 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.200068+0000", 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063743890000000003 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.200125+0000", 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7314e-05 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.200166+0000", 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0725000000000001e-05 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4579.0:3185 3.1b 3:d843d9ee:::rbd_data.11616dafc67f.0000000000000012:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:13.187304+0000", 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.710254838, 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013403892000000001, 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:15.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4579", 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3185 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187304+0000", 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187304+0000", 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187305+0000", 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1699999999999998e-07 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187308+0000", 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3790000000000002e-06 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:15.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187316+0000", 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8849999999999999e-06 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.187322+0000", 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6219999999999997e-06 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.192586+0000", 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0052640710000000004 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:15.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.192693+0000", 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107464 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.192905+0000", 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000211871 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.194039+0000", 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001133851 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.200598+0000", 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006559517 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.200663+0000", 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4821999999999997e-05 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:13.200708+0000", 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4793000000000003e-05 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.001 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.002 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:16.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:08:16.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:16.004 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:16.002+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:16.104 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:16.102+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:16.151 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone7 from 0xd625600 (overlap 0xd625600) to 0xab51000 2024-09-14T07:08:16.194 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:16.195 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:16.205 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:16.202+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:16.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:16.306 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:16.306+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:16.386 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:16.386 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:16.386 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:16.387 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:16.387 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:16.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:16.443 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:16.442+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:16.507 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:16.506+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:16.566 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:16.566 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:16.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:16.566 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:16.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.566 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4406.0:1168 3.2 3:42220d69:::rbd_data.11611b5ca054.000000000000002d:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3526656~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.529762814000001, 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014557588, 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4406", 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1168 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.567 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034026+0000", 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034029+0000", 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9270000000000001e-06 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034081+0000", 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1313000000000002e-05 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034115+0000", 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4156000000000003e-05 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034131+0000", 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6350000000000001e-05 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034276+0000", 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014510900000000001 2024-09-14T07:08:16.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034400+0000", 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012352899999999999 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.034616+0000", 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021601199999999999 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.045899+0000", 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011283519000000001 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048374+0000", 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024744379999999998 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048466+0000", 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1952000000000003e-05 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:07:54.048584+0000", 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011828300000000001 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.573 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:243 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.456924+0000", 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1068655070000002, 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016379797000000001, 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456924+0000", 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456924+0000", 2024-09-14T07:08:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456925+0000", 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5300000000000002e-07 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456929+0000", 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9169999999999999e-06 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456940+0000", 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1909999999999999e-05 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.456948+0000", 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7530000000000001e-06 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463086+0000", 2024-09-14T07:08:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0061378700000000001 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463225+0000", 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013878600000000001 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463974+0000", 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00074900399999999995 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.472880+0000", 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0089065640000000005 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473050+0000", 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016923900000000001 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473194+0000", 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014389200000000001 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473303+0000", 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010990899999999999 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:244 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.458447+0000", 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.105342168, 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019738037, 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 244 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458447+0000", 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458447+0000", 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458450+0000", 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8030000000000001e-06 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458465+0000", 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5440000000000001e-05 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458504+0000", 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8495999999999999e-05 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.458526+0000", 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1857000000000001e-05 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.466150+0000", 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0076242539999999996 2024-09-14T07:08:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.466381+0000", 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023085099999999999 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.467333+0000", 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095189000000000005 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475338+0000", 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080058879999999992 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.477979+0000", 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0026403799999999999 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478110+0000", 2024-09-14T07:08:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013101599999999999 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478185+0000", 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5161999999999996e-05 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:245 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.459256+0000", 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1045326979999999, 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019305170999999999, 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 245 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459256+0000", 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459256+0000", 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459259+0000", 2024-09-14T07:08:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459271+0000", 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-05 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459298+0000", 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6789999999999999e-05 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.459316+0000", 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7920000000000001e-05 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468338+0000", 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0090225199999999992 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468608+0000", 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026966799999999998 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469695+0000", 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010870420000000001 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475354+0000", 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056588259999999996 2024-09-14T07:08:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478294+0000", 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029405630000000002 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478415+0000", 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121036 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478562+0000", 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014641599999999999 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:246 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.460112+0000", 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1036773409999991, 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018752072000000002, 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 246 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460112+0000", 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460112+0000", 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460114+0000", 2024-09-14T07:08:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3029999999999998e-06 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460128+0000", 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3509999999999999e-05 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460180+0000", 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1910000000000003e-05 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460201+0000", 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1733e-05 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.470938+0000", 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01073671 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.471144+0000", 2024-09-14T07:08:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020582599999999999 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.471956+0000", 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00081192999999999999 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.477819+0000", 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0058631780000000001 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478701+0000", 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00088250899999999996 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478805+0000", 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010378300000000001 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478864+0000", 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8680000000000001e-05 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:247 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.460916+0000", 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1028735940000001, 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018220002999999999, 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 247 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460916+0000", 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460916+0000", 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460918+0000", 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.317e-06 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460933+0000", 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4876e-05 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460963+0000", 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0069999999999998e-05 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.460982+0000", 2024-09-14T07:08:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9596e-05 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473450+0000", 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012467974 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473685+0000", 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000234572 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.474344+0000", 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00065870800000000004 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478984+0000", 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046402270000000002 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479005+0000", 2024-09-14T07:08:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0704e-05 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479077+0000", 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2612000000000002e-05 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479136+0000", 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8347000000000002e-05 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:248 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.462023+0000", 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1017658929999996, 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02295055, 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 248 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462023+0000", 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462023+0000", 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462025+0000", 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.827e-06 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462035+0000", 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7599999999999997e-06 2024-09-14T07:08:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462056+0000", 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1387e-05 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.462069+0000", 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.311e-05 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475441+0000", 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013371686000000001 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475677+0000", 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000235832 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.476589+0000", 2024-09-14T07:08:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00091255600000000002 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483711+0000", 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0071217040000000004 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484699+0000", 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00098739599999999998 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484857+0000", 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015856600000000001 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484974+0000", 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011672599999999999 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:249 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.463149+0000", 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1006402580000003, 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022312041000000001, 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 249 2024-09-14T07:08:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463149+0000", 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463149+0000", 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463151+0000", 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463164+0000", 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3047000000000001e-05 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463197+0000", 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2790000000000003e-05 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.463218+0000", 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0639999999999999e-05 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479221+0000", 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016003632 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479350+0000", 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012889600000000001 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.479965+0000", 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00061449499999999995 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485152+0000", 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0051874069999999998 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485204+0000", 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-05 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485351+0000", 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014702199999999999 2024-09-14T07:08:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485461+0000", 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110429 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:250 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.464199+0000", 2024-09-14T07:08:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0995906529999999, 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026385663, 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 250 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464199+0000", 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464199+0000", 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464201+0000", 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2730000000000001e-06 2024-09-14T07:08:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464214+0000", 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3677e-05 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464249+0000", 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4496999999999999e-05 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.464275+0000", 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5948999999999998e-05 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.480967+0000", 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016691711000000001 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481101+0000", 2024-09-14T07:08:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000134206 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.482030+0000", 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00092936600000000005 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485642+0000", 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036116920000000001 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490383+0000", 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0047415640000000002 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490520+0000", 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000136372 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490584+0000", 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4356000000000005e-05 2024-09-14T07:08:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:251 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.465333+0000", 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0984559049999998, 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027876171000000002, 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 251 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465333+0000", 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465333+0000", 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465336+0000", 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6129999999999998e-06 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465371+0000", 2024-09-14T07:08:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.468e-05 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465400+0000", 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9563000000000001e-05 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.465420+0000", 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0302999999999999e-05 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485814+0000", 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020393593000000002 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486067+0000", 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025264100000000001 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487153+0000", 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001085899 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.492983+0000", 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0058306779999999997 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493005+0000", 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1759999999999998e-05 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493092+0000", 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6599000000000002e-05 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493209+0000", 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117842 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:295 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0941890109999992, 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028784614, 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 295 2024-09-14T07:08:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469603+0000", 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469619+0000", 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6163000000000001e-05 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469646+0000", 2024-09-14T07:08:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6040000000000001e-05 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469660+0000", 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4843e-05 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488319+0000", 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018658726 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488567+0000", 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024742500000000001 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.489393+0000", 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082641099999999999 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493423+0000", 2024-09-14T07:08:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040299050000000003 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498065+0000", 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046422310000000001 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498254+0000", 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000189278 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498385+0000", 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013043899999999999 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:296 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.473652+0000", 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0901374189999995, 2024-09-14T07:08:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.025354419, 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 296 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473652+0000", 2024-09-14T07:08:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473652+0000", 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473655+0000", 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7999999999999999e-06 2024-09-14T07:08:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473670+0000", 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5153e-05 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473736+0000", 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6270000000000001e-05 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.473758+0000", 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2405999999999999e-05 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490685+0000", 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016926566000000001 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.490826+0000", 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000140667 2024-09-14T07:08:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.491817+0000", 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0009915010000000001 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498557+0000", 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006739798 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498749+0000", 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019212 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498885+0000", 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013625900000000001 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499006+0000", 2024-09-14T07:08:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012087900000000001 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:298 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.475423+0000", 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0883656760000004, 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023859328999999999, 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 298 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475423+0000", 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475423+0000", 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475426+0000", 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.12e-06 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475437+0000", 2024-09-14T07:08:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1783000000000001e-05 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475463+0000", 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5746e-05 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.475482+0000", 2024-09-14T07:08:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.906e-05 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493548+0000", 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018066181000000001 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493806+0000", 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025763499999999998 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.494898+0000", 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001091921 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498716+0000", 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0038182950000000002 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499050+0000", 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033355199999999998 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499174+0000", 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012412499999999999 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499283+0000", 2024-09-14T07:08:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108911 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:300 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.478508+0000", 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0852810900000005, 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026193111000000002, 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 300 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478508+0000", 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478508+0000", 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478510+0000", 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.317e-06 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478524+0000", 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.366e-05 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478552+0000", 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7835999999999999e-05 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.478571+0000", 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9466e-05 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.496068+0000", 2024-09-14T07:08:16.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017496989000000001 2024-09-14T07:08:16.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.496336+0000", 2024-09-14T07:08:16.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026767400000000003 2024-09-14T07:08:16.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.497277+0000", 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00094107000000000001 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504404+0000", 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0071268549999999996 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504494+0000", 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0532000000000007e-05 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504602+0000", 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010760999999999999 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504701+0000", 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9102000000000001e-05 2024-09-14T07:08:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:306 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.481121+0000", 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0826681370000006, 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023939642000000001, 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 306 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481121+0000", 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481121+0000", 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481124+0000", 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481138+0000", 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4574e-05 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481173+0000", 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4069999999999997e-05 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.481197+0000", 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4243000000000002e-05 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499406+0000", 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018209672999999999 2024-09-14T07:08:16.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.499598+0000", 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019118800000000001 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.500323+0000", 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000724879 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504768+0000", 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044458279999999998 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504871+0000", 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102222 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504948+0000", 2024-09-14T07:08:16.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7146999999999998e-05 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.505061+0000", 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112942 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:310 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.484907+0000", 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0788819759999999, 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023074125000000001, 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 310 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484907+0000", 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484907+0000", 2024-09-14T07:08:16.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484910+0000", 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484924+0000", 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3923e-05 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484955+0000", 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1309999999999997e-05 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484975+0000", 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9792999999999998e-05 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.501562+0000", 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016586825999999999 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.501825+0000", 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026327399999999997 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.502909+0000", 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001083922 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507438+0000", 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045286409999999999 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507572+0000", 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000134139 2024-09-14T07:08:16.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507884+0000", 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031209800000000002 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.507981+0000", 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7509000000000004e-05 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.677 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:315 3.b 3:d73f643a:::rbd_data.1161dbabc1ae.0000000000000008:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.486826+0000", 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0769632009999999, 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017581507999999999, 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 315 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486826+0000", 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486826+0000", 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486829+0000", 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6570000000000001e-06 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486841+0000", 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2830000000000001e-05 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486876+0000", 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4585999999999997e-05 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486897+0000", 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.09e-05 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.503971+0000", 2024-09-14T07:08:16.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017073748 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504162+0000", 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019167199999999999 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.504407+0000", 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024511500000000001 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:316 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.488078+0000", 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.075711664, 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021237334, 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 316 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.683 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488078+0000", 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488078+0000", 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488081+0000", 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3459999999999998e-06 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488095+0000", 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4273999999999999e-05 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488152+0000", 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6784999999999999e-05 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488173+0000", 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0653999999999999e-05 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.505184+0000", 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017010952999999999 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.505351+0000", 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000167508 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.506477+0000", 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126242 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.508648+0000", 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021702829999999998 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.509219+0000", 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00057135599999999997 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.509274+0000", 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4920999999999998e-05 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.509315+0000", 2024-09-14T07:08:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1012e-05 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4649.0:661 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:16.271625+0000", 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.29216437899999997, 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020583471999999998, 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4649", 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1362242474", 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 661 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.271625+0000", 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.271625+0000", 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.271626+0000", 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.31e-06 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272902+0000", 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001275871 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272915+0000", 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3042e-05 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272921+0000", 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1800000000000001e-06 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272984+0000", 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2966000000000007e-05 2024-09-14T07:08:16.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.273091+0000", 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107036 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.273339+0000", 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024731800000000001 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.281271+0000", 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079322130000000005 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.292074+0000", 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010802934 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.292153+0000", 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9246999999999995e-05 2024-09-14T07:08:16.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.292208+0000", 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5355000000000001e-05 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4649.0:662 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:head [truncate 13893632] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:16.272938+0000", 2024-09-14T07:08:16.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.29085104299999998, 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019510255000000001, 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4649", 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1362242474", 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 662 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272938+0000", 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272938+0000", 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272939+0000", 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7199999999999998e-07 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272942+0000", 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272958+0000", 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6069000000000001e-05 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.272965+0000", 2024-09-14T07:08:16.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3209999999999999e-06 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.281871+0000", 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0089060790000000008 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.281961+0000", 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9803999999999996e-05 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.282160+0000", 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019849399999999999 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.283800+0000", 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001640208 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.292339+0000", 2024-09-14T07:08:16.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0085388930000000005 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.292402+0000", 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3658000000000006e-05 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:16.292448+0000", 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6013000000000001e-05 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.701 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:16.702 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:16.702 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:16.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:16.704 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:16.702+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:16.804 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:16.802+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:16.882 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:16.882 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:16.882 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:16.882 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:16.897 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:16.905 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:16.902+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:16.979 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:08:16.989 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T07:08:17.006 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:17.002+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:17.037 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:17.038 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:17.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:17.106 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:17.102+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:17.148 INFO:teuthology.orchestra.run.smithi145.stdout:804 read 0xd3d6c00 thru 0xd3d8603 (0x1a04 bytes) 2024-09-14T07:08:17.149 INFO:teuthology.orchestra.run.smithi145.stdout:805 punch from 0xcecb400 to 0xced8000, (0xcc00 bytes) 2024-09-14T07:08:17.158 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:17.158 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:17.159 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T07:08:17.159 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.297712 2024-09-14T07:08:17.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.2977115458361327 2024-09-14T07:08:17.192 INFO:teuthology.orchestra.run.smithi145.stdout:806 read 0xaeca00 thru 0xaf7fa8 (0xb5a9 bytes) 2024-09-14T07:08:17.193 INFO:teuthology.orchestra.run.smithi145.stdout:807 read 0x75fee00 thru 0x760b571 (0xc772 bytes) 2024-09-14T07:08:17.193 INFO:teuthology.orchestra.run.smithi145.stdout:810 write 0x26f1600 thru 0x26fa00c (0x8a0d bytes) 2024-09-14T07:08:17.194 INFO:teuthology.orchestra.run.smithi145.stdout:811 read 0xd067600 thru 0xd06f3eb (0x7dec bytes) 2024-09-14T07:08:17.195 INFO:teuthology.orchestra.run.smithi145.stdout:812 write 0xbda7e00 thru 0xbdb7cb2 (0xfeb3 bytes) 2024-09-14T07:08:17.195 INFO:teuthology.orchestra.run.smithi145.stdout:813 write 0xb042000 thru 0xb048a02 (0x6a03 bytes) 2024-09-14T07:08:17.195 INFO:teuthology.orchestra.run.smithi145.stdout:815 write 0xd13fc00 thru 0xd14a24a (0xa64b bytes) 2024-09-14T07:08:17.195 INFO:teuthology.orchestra.run.smithi145.stdout:817 read 0x36d3400 thru 0x36de472 (0xb073 bytes) 2024-09-14T07:08:17.196 INFO:teuthology.orchestra.run.smithi145.stdout:818 write 0xc181000 thru 0xc1822c6 (0x12c7 bytes) 2024-09-14T07:08:17.196 INFO:teuthology.orchestra.run.smithi145.stdout:819 write 0x5177200 thru 0x518030b (0x910c bytes) 2024-09-14T07:08:17.198 INFO:teuthology.orchestra.run.smithi145.stdout:822 write 0x83dd800 thru 0x83eb23c (0xda3d bytes) 2024-09-14T07:08:17.199 INFO:teuthology.orchestra.run.smithi145.stdout:824 write 0xcfe800 thru 0xd0785b (0x905c bytes) 2024-09-14T07:08:17.200 INFO:teuthology.orchestra.run.smithi145.stdout:825 read 0x9ea2a00 thru 0x9ea78b8 (0x4eb9 bytes) 2024-09-14T07:08:17.200 INFO:teuthology.orchestra.run.smithi145.stdout:826 write 0x207aa00 thru 0x207c29f (0x18a0 bytes) 2024-09-14T07:08:17.201 INFO:teuthology.orchestra.run.smithi145.stdout:828 clone 9 order 25 su 4194304 sc 5 2024-09-14T07:08:17.207 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:17.206+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:832 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106514+0000", 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.141452608, 2024-09-14T07:08:17.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047025360000000002, 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 832 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106514+0000", 2024-09-14T07:08:17.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106514+0000", 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106517+0000", 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2619999999999999e-06 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106533+0000", 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1824999999999999e-05 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106541+0000", 2024-09-14T07:08:17.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5279999999999998e-06 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153376+0000", 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046835379000000003 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153450+0000", 2024-09-14T07:08:17.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3460999999999993e-05 2024-09-14T07:08:17.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153539+0000", 2024-09-14T07:08:17.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9641000000000005e-05 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1611 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.141444225000001, 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047250024000000002, 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.255 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1611 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106522+0000", 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106523+0000", 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106535+0000", 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1606e-05 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106544+0000", 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7369999999999995e-06 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106548+0000", 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5379999999999996e-06 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153573+0000", 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047024375 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153654+0000", 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1230000000000007e-05 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153772+0000", 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118558 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1612 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.141283966, 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047314770999999999, 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1612 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106683+0000", 2024-09-14T07:08:17.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9599999999999998e-07 2024-09-14T07:08:17.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106687+0000", 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7759999999999999e-06 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106696+0000", 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0129999999999999e-06 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4431e-05 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153824+0000", 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047113006999999998 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153901+0000", 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7015999999999996e-05 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.153997+0000", 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6731999999999995e-05 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:833 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:17.265 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.141255498, 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047546469000000001, 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 833 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.266 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106711+0000", 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106712+0000", 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.043e-06 2024-09-14T07:08:17.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106716+0000", 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3579999999999996e-06 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106728+0000", 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1534e-05 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106735+0000", 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2740000000000002e-06 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154044+0000", 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047308463000000002 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154141+0000", 2024-09-14T07:08:17.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6903000000000004e-05 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154258+0000", 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116894 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:834 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.141135029999999, 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047638265999999999, 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 834 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106832+0000", 2024-09-14T07:08:17.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5700000000000002e-07 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106836+0000", 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0489999999999997e-06 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106854+0000", 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7632e-05 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106862+0000", 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154302+0000", 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047440309 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154375+0000", 2024-09-14T07:08:17.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2943999999999998e-05 2024-09-14T07:08:17.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154470+0000", 2024-09-14T07:08:17.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4829999999999998e-05 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1613 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.106902+0000", 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.141064427, 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047815349, 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1613 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106902+0000", 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106902+0000", 2024-09-14T07:08:17.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106903+0000", 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5600000000000005e-07 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106906+0000", 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106914+0000", 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0990000000000001e-06 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.106918+0000", 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5050000000000001e-06 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154517+0000", 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047598616000000003 2024-09-14T07:08:17.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154600+0000", 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2868999999999996e-05 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154718+0000", 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117973 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1614 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107048+0000", 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140918716, 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047890478, 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1614 2024-09-14T07:08:17.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107048+0000", 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107048+0000", 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107049+0000", 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4200000000000005e-07 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107053+0000", 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2309999999999998e-06 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107063+0000", 2024-09-14T07:08:17.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9049999999999992e-06 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107070+0000", 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6610000000000003e-06 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154768+0000", 2024-09-14T07:08:17.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047698590999999999 2024-09-14T07:08:17.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154840+0000", 2024-09-14T07:08:17.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2015999999999996e-05 2024-09-14T07:08:17.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154938+0000", 2024-09-14T07:08:17.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8232000000000004e-05 2024-09-14T07:08:17.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.286 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:835 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107120+0000", 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140846663, 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048072937000000003, 2024-09-14T07:08:17.287 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 835 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107120+0000", 2024-09-14T07:08:17.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107120+0000", 2024-09-14T07:08:17.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107121+0000", 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.015e-06 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107125+0000", 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4800000000000003e-06 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107137+0000", 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1542000000000001e-05 2024-09-14T07:08:17.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107144+0000", 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3919999999999997e-06 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.154985+0000", 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047840420000000002 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155080+0000", 2024-09-14T07:08:17.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4868e-05 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155193+0000", 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011322 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1615 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107193+0000", 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140773597999999, 2024-09-14T07:08:17.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048212334000000003, 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1615 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107193+0000", 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107193+0000", 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107194+0000", 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1200000000000003e-07 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107197+0000", 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4249999999999998e-06 2024-09-14T07:08:17.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107204+0000", 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3930000000000002e-06 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107210+0000", 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5500000000000002e-06 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155237+0000", 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048026712999999999 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155314+0000", 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6936999999999999e-05 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155405+0000", 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1704000000000002e-05 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.296 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:836 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107211+0000", 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140755178999999, 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048451056999999999, 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 836 2024-09-14T07:08:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107211+0000", 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107211+0000", 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107212+0000", 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.037e-06 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107217+0000", 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107227+0000", 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.024e-05 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107234+0000", 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1239999999999999e-06 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155456+0000", 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048221676999999998 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155539+0000", 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2878000000000002e-05 2024-09-14T07:08:17.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155662+0000", 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000123681 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:837 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140524999, 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048434352, 2024-09-14T07:08:17.301 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.302 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.302 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.302 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 837 2024-09-14T07:08:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107442+0000", 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2900000000000003e-07 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107447+0000", 2024-09-14T07:08:17.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107462+0000", 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4948e-05 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107469+0000", 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6830000000000008e-06 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155707+0000", 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048237774999999997 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155783+0000", 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155876+0000", 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3113000000000001e-05 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1616 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140412020999999, 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048566559000000002, 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1616 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107555+0000", 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8100000000000002e-07 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107559+0000", 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3799999999999998e-06 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107567+0000", 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.994e-06 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107571+0000", 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6959999999999996e-06 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.155923+0000", 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048351466000000003 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156005+0000", 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2137000000000003e-05 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156121+0000", 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116205 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1617 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107603+0000", 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140363109000001, 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048741325000000002, 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1617 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107603+0000", 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107603+0000", 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107604+0000", 2024-09-14T07:08:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8800000000000002e-07 2024-09-14T07:08:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107608+0000", 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4860000000000002e-06 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107616+0000", 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8040000000000007e-06 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107631+0000", 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4511999999999999e-05 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156181+0000", 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048549595000000001 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156252+0000", 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1250999999999999e-05 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156345+0000", 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2989e-05 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:838 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107605+0000", 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140361304000001, 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048990005000000003, 2024-09-14T07:08:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 838 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107605+0000", 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107605+0000", 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107606+0000", 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107611+0000", 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107632+0000", 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1202000000000001e-05 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107641+0000", 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1750000000000005e-06 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156394+0000", 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048753071000000002 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156477+0000", 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3543999999999998e-05 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156595+0000", 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118053 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:839 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107741+0000", 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140226001, 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049065575, 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 839 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107741+0000", 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107741+0000", 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107742+0000", 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9099999999999991e-07 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107747+0000", 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9159999999999998e-06 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107759+0000", 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1544e-05 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107770+0000", 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156640+0000", 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048869888 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156714+0000", 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3981999999999997e-05 2024-09-14T07:08:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156806+0000", 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2567999999999999e-05 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1618 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107758+0000", 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.140208394, 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049297046999999997, 2024-09-14T07:08:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1618 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107758+0000", 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107758+0000", 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107759+0000", 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.251e-06 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107764+0000", 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.746e-06 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107773+0000", 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1409999999999994e-06 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107780+0000", 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4529999999999999e-06 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156852+0000", 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049072431 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.156934+0000", 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1589999999999996e-05 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157055+0000", 2024-09-14T07:08:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012143500000000001 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1619 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.139993079, 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049542967, 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1619 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.61e-07 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107986+0000", 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2136e-05 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107995+0000", 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9779999999999994e-06 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108002+0000", 2024-09-14T07:08:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0149999999999998e-06 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157318+0000", 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049315921999999998 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157399+0000", 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0793999999999999e-05 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157516+0000", 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117361 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:840 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.139992479, 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049296567999999999, 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 840 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107974+0000", 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107975+0000", 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.246e-06 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107980+0000", 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7809999999999996e-06 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107992+0000", 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1415e-05 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.107999+0000", 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4259999999999997e-06 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157108+0000", 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04910867 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157181+0000", 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2887999999999997e-05 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157271+0000", 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0142000000000005e-05 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4498.0:841 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.139706053999999, 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049470640000000003, 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4498", 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/710615182", 2024-09-14T07:08:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 841 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108261+0000", 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999998e-07 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108269+0000", 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108282+0000", 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2318e-05 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108289+0000", 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6729999999999992e-06 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157564+0000", 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049274577999999999 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157641+0000", 2024-09-14T07:08:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6814e-05 2024-09-14T07:08:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157731+0000", 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:1620 3.8 3:162f5a10:::rbd_header.116148d60084:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:04.108352+0000", 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.139614991, 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049620367999999998, 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:08:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1620 2024-09-14T07:08:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108352+0000", 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108352+0000", 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108353+0000", 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1900000000000001e-07 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108357+0000", 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4240000000000003e-06 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108369+0000", 2024-09-14T07:08:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2269999999999999e-05 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.108376+0000", 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157776+0000", 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049399817999999998 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157859+0000", 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2640999999999997e-05 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:04.157972+0000", 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113074 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:17.351 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:17.352 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:17.353 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:17.349+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:17.453 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:17.449+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:17.471 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:17.471 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:17.471 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:17.485 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:17.571 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:17.569+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:17.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:17.652 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:17.652 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:17.652 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:17.655 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:17.649+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:17.666 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:17.756 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:17.753+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:17.794 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:17.811 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:17.811 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:17.831 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:17.831 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.538484947000001, 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:17.842 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:17.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.848 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1502 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61812058700000005, 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042080580999999999, 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1502 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220869+0000", 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2300000000000001e-06 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220888+0000", 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8607e-05 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220907+0000", 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9422000000000001e-05 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220922+0000", 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4375999999999999e-05 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262777+0000", 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855537999999998 2024-09-14T07:08:17.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262833+0000", 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5729000000000001e-05 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262949+0000", 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115679 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1503 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61804577500000002, 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042296750000000001, 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1503 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220945+0000", 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.013e-06 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220951+0000", 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.806e-06 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220970+0000", 2024-09-14T07:08:17.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8649999999999999e-05 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3964999999999999e-05 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262989+0000", 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042005845 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263116+0000", 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012688 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263240+0000", 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012359099999999999 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1504 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61798665100000005, 2024-09-14T07:08:17.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042493942999999999, 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1504 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.063e-06 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221011+0000", 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8059999999999997e-06 2024-09-14T07:08:17.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221030+0000", 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8652999999999999e-05 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221044+0000", 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3879999999999999e-05 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263275+0000", 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042231869999999998 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263365+0000", 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9813000000000002e-05 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263496+0000", 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000130858 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1634 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.614792589, 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041961863000000002, 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1634 2024-09-14T07:08:17.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224198+0000", 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.77e-06 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224205+0000", 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6429999999999998e-06 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224240+0000", 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5714999999999999e-05 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224555+0000", 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031487300000000002 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265911+0000", 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041355780000000002 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266040+0000", 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012856099999999999 2024-09-14T07:08:17.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266158+0000", 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118521 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1635 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224626+0000", 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61436252099999999, 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041935817, 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1635 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224626+0000", 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224626+0000", 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224628+0000", 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.99e-06 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224637+0000", 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1370000000000008e-06 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224687+0000", 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9910999999999997e-05 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224706+0000", 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8298999999999999e-05 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266208+0000", 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041502451000000003 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266272+0000", 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4404000000000002e-05 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266562+0000", 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028962500000000001 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1636 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.871 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61425896400000002, 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042143574000000003, 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1636 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224731+0000", 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5600000000000001e-06 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224738+0000", 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224756+0000", 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8153999999999999e-05 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224770+0000", 2024-09-14T07:08:17.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4168999999999999e-05 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266618+0000", 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041847662000000001 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266730+0000", 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000111993 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266873+0000", 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014374400000000001 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1643 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 8978432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61317792900000001, 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042103224000000002, 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1643 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225812+0000", 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.207e-06 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225818+0000", 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225836+0000", 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8221999999999999e-05 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225849+0000", 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.358e-05 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267705+0000", 2024-09-14T07:08:17.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855723999999997 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267805+0000", 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9437000000000004e-05 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267914+0000", 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109554 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1644 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61311970100000002, 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042307838, 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1644 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.881 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225870+0000", 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2270000000000001e-06 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225876+0000", 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2569999999999998e-06 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225894+0000", 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8385000000000001e-05 2024-09-14T07:08:17.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225936+0000", 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2159000000000001e-05 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267952+0000", 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04201585 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268067+0000", 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114985 2024-09-14T07:08:17.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268177+0000", 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109975 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1645 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61303132400000004, 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042484558999999998, 2024-09-14T07:08:17.885 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1645 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225959+0000", 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225978+0000", 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9466e-05 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225998+0000", 2024-09-14T07:08:17.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9315999999999999e-05 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226013+0000", 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268213+0000", 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200410000000001 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268324+0000", 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011123799999999999 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268442+0000", 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117889 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.889 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1646 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.612954946, 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042595283999999997, 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1646 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226035+0000", 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5060000000000001e-06 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226042+0000", 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4370000000000004e-06 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226060+0000", 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8266000000000001e-05 2024-09-14T07:08:17.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226074+0000", 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4029000000000001e-05 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:17.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042405023999999999 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268551+0000", 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1973999999999999e-05 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268629+0000", 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8047999999999995e-05 2024-09-14T07:08:17.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1647 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12910592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61289691199999996, 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042800796000000002, 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1647 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226093+0000", 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.46e-06 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226099+0000", 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3530000000000002e-06 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226118+0000", 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9055999999999998e-05 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226132+0000", 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268673+0000", 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042541272999999998 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268775+0000", 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102302 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268893+0000", 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117539 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1648 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61283777100000003, 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042904468000000001, 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1648 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226152+0000", 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.234e-06 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226157+0000", 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2000000000000002e-06 2024-09-14T07:08:17.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226175+0000", 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7059e-05 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226204+0000", 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9536e-05 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268938+0000", 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042733698000000001 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268992+0000", 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4693e-05 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269056+0000", 2024-09-14T07:08:17.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3047999999999997e-05 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1649 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61276356300000001, 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043088794999999999, 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1649 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226227+0000", 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.33e-06 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226232+0000", 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.733e-06 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226251+0000", 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8816000000000001e-05 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226264+0000", 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-05 2024-09-14T07:08:17.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269097+0000", 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0428324 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269197+0000", 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100419 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269314+0000", 2024-09-14T07:08:17.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116897 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1651 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61270471500000001, 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043199128000000003, 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1651 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:17.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226285+0000", 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.237e-06 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226291+0000", 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1789999999999999e-06 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226311+0000", 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0656999999999999e-05 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226327+0000", 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5948999999999999e-05 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269352+0000", 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043024830999999999 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269419+0000", 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7281999999999997e-05 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269483+0000", 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3992999999999995e-05 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1700 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 32768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:17.914 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61264138700000004, 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043417208999999998, 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1700 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226349+0000", 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3629999999999999e-06 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226354+0000", 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:08:17.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226372+0000", 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7612999999999999e-05 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226386+0000", 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4146e-05 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269525+0000", 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043139380999999997 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269631+0000", 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105574 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269765+0000", 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133759 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1702 3.13 3:cfeabdd5:::rbd_data.11616dafc67f.000000000000000f:9 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:17.918 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61255296699999995, 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043560261000000003, 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1702 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226437+0000", 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.567e-06 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226457+0000", 2024-09-14T07:08:17.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395999999999999e-05 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226477+0000", 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9998999999999999e-05 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226491+0000", 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4503e-05 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269835+0000", 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043343995000000003 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269914+0000", 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9018999999999997e-05 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269996+0000", 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1781999999999997e-05 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1740 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.61247607299999995, 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043673828999999997, 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1740 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226514+0000", 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4869999999999999e-06 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226520+0000", 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.976e-06 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226538+0000", 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7972999999999999e-05 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226551+0000", 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3142999999999999e-05 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270041+0000", 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043490174999999999 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270101+0000", 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9515999999999997e-05 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270187+0000", 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5558999999999999e-05 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.926 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.54770181200000001, 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:08:17.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:08:17.932 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4803.0:1903 3.1f e43/21 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head v 43'21, mlcod=42'19)", 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.53054099300000002, 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.057064131999999997, 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:17.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308449+0000", 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.279e-06 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324156+0000", 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015707199000000002 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324195+0000", 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9070999999999999e-05 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324217+0000", 2024-09-14T07:08:17.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1752000000000001e-05 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324333+0000", 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011607000000000001 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324359+0000", 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6296e-05 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:17.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365477+0000", 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041117065000000001 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365512+0000", 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-05 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:17.937 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:17.938 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:17.939 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:17.940 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:17.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:17.940 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:17.933+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:17.940 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.0 primary-affinity to 0.297712 (8195102) 2024-09-14T07:08:18.039 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:18.037+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:18.111 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:18.111 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:18.124 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:18.140 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:18.138+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:18.241 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:18.237+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:18.292 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:18.293 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:18.306 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:18.342 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:18.338+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:18.443 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:18.441+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:18.468 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:18.468 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:18.482 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:18.544 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:18.542+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:18.645 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:18.641+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:18.647 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:18.647 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:18.661 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:18.745 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:18.742+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:18.825 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:18.825 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:18.840 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:18.887 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:18.885+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:18.947 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:18.946+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:18.964 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:18.964 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:18.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:19.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:08:19.048 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:19.045+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:19.128 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:19.128 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:19.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:19.149 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:19.145+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:19.176 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone8 from 0xd625600 (overlap 0xd625600) to 0x6b12a00 2024-09-14T07:08:19.253 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:19.246+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:19.294 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:19.294 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:19.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:19.351 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:19.350+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:19.452 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:19.450+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:19.480 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:19.480 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:19.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:19.553 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:19.549+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:19.654 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:19.650+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:19.669 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:19.669 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:19.685 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:19.754 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:19.749+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:19.847 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:19.847 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:19.855 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:19.854+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:19.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:19.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:19.957 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:19.953+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:20.005 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:08:20.039 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:20.039 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:20.050 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T07:08:20.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:20.102 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:20.102+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:20.106 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:20.106 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:20.123 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:20.123 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:20.140 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:20.141 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:20.158 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:20.153+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:20.162 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:20.162 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:20.183 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:20.204 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:20.219 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:20.219 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:20.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:20.279 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:20.278+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:20.361 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:20.357+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:20.404 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:20.404 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:20.404 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:20.404 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:20.405 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:20.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:20.462 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:20.457+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:20.558 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1537 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 9371648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220923+0000", 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3357078950000001, 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054023002000000001, 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1537 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220923+0000", 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220923+0000", 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220924+0000", 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4799999999999999e-07 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220926+0000", 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.419e-06 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220933+0000", 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.725e-06 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220938+0000", 2024-09-14T07:08:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0849999999999996e-06 2024-09-14T07:08:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274848+0000", 2024-09-14T07:08:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053909855 2024-09-14T07:08:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274891+0000", 2024-09-14T07:08:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3148000000000001e-05 2024-09-14T07:08:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274946+0000", 2024-09-14T07:08:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5322000000000003e-05 2024-09-14T07:08:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1538 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3356852410000002, 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054129517000000002, 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1538 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220949+0000", 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-06 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220959+0000", 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0932e-05 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220965+0000", 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.49e-06 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274976+0000", 2024-09-14T07:08:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054011221999999998 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275019+0000", 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2889000000000002e-05 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275076+0000", 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6455000000000001e-05 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1539 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3356585249999999, 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054229843, 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1539 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220976+0000", 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-06 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1069999999999999e-06 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220988+0000", 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0119999999999996e-06 2024-09-14T07:08:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275106+0000", 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054118053999999999 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275149+0000", 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2830000000000002e-05 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275203+0000", 2024-09-14T07:08:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4044e-05 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1540 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.335636209, 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05433267, 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1540 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220996+0000", 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220998+0000", 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9759999999999998e-06 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:08:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221009+0000", 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.242e-06 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275233+0000", 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054223461000000001 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275275+0000", 2024-09-14T07:08:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2726999999999999e-05 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275328+0000", 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2332e-05 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1541 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3356140129999998, 2024-09-14T07:08:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054437230000000003, 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1541 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221018+0000", 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2e-07 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221020+0000", 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3530000000000002e-06 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221026+0000", 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3110000000000002e-06 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221031+0000", 2024-09-14T07:08:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275358+0000", 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054326722000000001 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275401+0000", 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2849000000000003e-05 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275454+0000", 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3844000000000002e-05 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1542 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3355935990000001, 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054541385999999997, 2024-09-14T07:08:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1542 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3700000000000001e-07 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221040+0000", 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8059999999999999e-06 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221046+0000", 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1009999999999997e-06 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221051+0000", 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6940000000000003e-06 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275484+0000", 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054433003000000001 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275527+0000", 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3031999999999997e-05 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275579+0000", 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2413000000000001e-05 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1543 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3355738430000001, 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054648398000000001, 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1543 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221058+0000", 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4299999999999999e-07 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221060+0000", 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7990000000000001e-06 2024-09-14T07:08:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221066+0000", 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221070+0000", 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7010000000000004e-06 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275609+0000", 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054538968 2024-09-14T07:08:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275652+0000", 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2864000000000002e-05 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275706+0000", 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3597000000000002e-05 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1545 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3355541770000001, 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054855590000000003, 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1545 2024-09-14T07:08:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4499999999999998e-07 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221085+0000", 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0330000000000008e-06 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221092+0000", 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.793e-06 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221098+0000", 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3090000000000003e-06 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275744+0000", 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054646404000000003 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275821+0000", 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6793999999999997e-05 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275933+0000", 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111912 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1583 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 524288~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.335526014, 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055023028000000002, 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1583 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221106+0000", 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221108+0000", 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0229999999999999e-06 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221121+0000", 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3645000000000001e-05 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221127+0000", 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.541e-06 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275735+0000", 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054608372000000002 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275817+0000", 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2083999999999995e-05 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276128+0000", 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031090000000000002 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1585 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3354961799999998, 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055153874999999998, 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1585 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221136+0000", 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7300000000000001e-07 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221138+0000", 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221144+0000", 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5100000000000004e-06 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221149+0000", 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.843e-06 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276165+0000", 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055015677999999998 2024-09-14T07:08:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276231+0000", 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5959999999999999e-05 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276289+0000", 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8100000000000003e-05 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1586 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.335472566, 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055262004000000003, 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1586 2024-09-14T07:08:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221166+0000", 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3180000000000001e-06 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221173+0000", 2024-09-14T07:08:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221178+0000", 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2050000000000001e-06 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276321+0000", 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055142663000000001 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276364+0000", 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3615000000000002e-05 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276421+0000", 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6453000000000003e-05 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1588 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3354453290000001, 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055364653999999999, 2024-09-14T07:08:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1588 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221189+0000", 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3269999999999999e-06 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221195+0000", 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2169999999999999e-06 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221200+0000", 2024-09-14T07:08:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276451+0000", 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055251452999999999 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276497+0000", 2024-09-14T07:08:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5114000000000002e-05 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276551+0000", 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4079999999999997e-05 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1589 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3354246650000001, 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055473737000000002, 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1589 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:08:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221209+0000", 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9269999999999999e-06 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221215+0000", 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3069999999999999e-06 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221220+0000", 2024-09-14T07:08:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6369999999999998e-06 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276582+0000", 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055361856000000001 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276625+0000", 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2803000000000003e-05 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276680+0000", 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5789999999999999e-05 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1591 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.335404305, 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055584770999999998, 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1591 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222763+0000", 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001535914 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222789+0000", 2024-09-14T07:08:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6194e-05 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222862+0000", 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2529999999999998e-05 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222953+0000", 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1185000000000002e-05 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276713+0000", 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053760371000000001 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276758+0000", 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4932000000000003e-05 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276812+0000", 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3644999999999999e-05 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1593 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 6225920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.33363363, 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054086803000000003, 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1593 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223000+0000", 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.289e-06 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223007+0000", 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.058e-06 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223013+0000", 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5779999999999998e-06 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276973+0000", 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053960263000000001 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277028+0000", 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000002e-05 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277084+0000", 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6357999999999998e-05 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1594 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3336097649999998, 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054193419, 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1594 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223035+0000", 2024-09-14T07:08:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2994e-05 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223043+0000", 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6539999999999999e-06 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223048+0000", 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6149999999999996e-06 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277114+0000", 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054066256 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277157+0000", 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2629000000000001e-05 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277215+0000", 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7803999999999997e-05 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1597 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223595+0000", 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3330365560000002, 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054009680999999997, 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1597 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223595+0000", 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223595+0000", 2024-09-14T07:08:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223595+0000", 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2100000000000002e-07 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223613+0000", 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7748999999999999e-05 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223624+0000", 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1460999999999999e-05 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223631+0000", 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3369999999999996e-06 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277507+0000", 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053876214999999998 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277550+0000", 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2704000000000003e-05 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277604+0000", 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4793999999999997e-05 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1598 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.3329902100000002, 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054092236000000002, 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1598 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224174+0000", 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000533265 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224179+0000", 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2420000000000002e-06 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224190+0000", 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-05 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224197+0000", 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3549999999999999e-06 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277635+0000", 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053437314999999999 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277679+0000", 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4628e-05 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277733+0000", 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4131000000000001e-05 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.2563245740000002, 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.2113342679999999, 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:08:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:08:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:08:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:08:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:20.655 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:20.657 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:20.657 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:20.654+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:20.758 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:20.754+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:20.792 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:20.793 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:20.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:20.859 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:20.858+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:20.934 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:20.934 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:20.934 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:20.934 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:20.934 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:20.950 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:20.995 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:20.993+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:21.061 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:21.057+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:21.082 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:21.082 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:21.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.779217836000001, 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.083 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:21.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:21.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:21.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:21.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:21.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.089 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1502 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8588534760000002, 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042080580999999999, 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1502 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220869+0000", 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2300000000000001e-06 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220888+0000", 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8607e-05 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220907+0000", 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9422000000000001e-05 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220922+0000", 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4375999999999999e-05 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262777+0000", 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855537999999998 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262833+0000", 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5729000000000001e-05 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262949+0000", 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115679 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.093 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1503 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8587786639999999, 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042296750000000001, 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1503 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.094 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220945+0000", 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.013e-06 2024-09-14T07:08:21.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220951+0000", 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.806e-06 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220970+0000", 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8649999999999999e-05 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:21.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3964999999999999e-05 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262989+0000", 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042005845 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263116+0000", 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012688 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263240+0000", 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012359099999999999 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1504 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8587195400000001, 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042493942999999999, 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1504 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.063e-06 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221011+0000", 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8059999999999997e-06 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221030+0000", 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8652999999999999e-05 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221044+0000", 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3879999999999999e-05 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263275+0000", 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042231869999999998 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263365+0000", 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9813000000000002e-05 2024-09-14T07:08:21.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263496+0000", 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000130858 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1634 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8555254780000001, 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041961863000000002, 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.102 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1634 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:21.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224198+0000", 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.77e-06 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224205+0000", 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6429999999999998e-06 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224240+0000", 2024-09-14T07:08:21.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5714999999999999e-05 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224555+0000", 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031487300000000002 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265911+0000", 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041355780000000002 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266040+0000", 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012856099999999999 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266158+0000", 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118521 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1635 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224626+0000", 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8550954100000001, 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041935817, 2024-09-14T07:08:21.106 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1635 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224626+0000", 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224626+0000", 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224628+0000", 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.99e-06 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224637+0000", 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1370000000000008e-06 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224687+0000", 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9910999999999997e-05 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224706+0000", 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8298999999999999e-05 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266208+0000", 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041502451000000003 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266272+0000", 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4404000000000002e-05 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266562+0000", 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028962500000000001 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1636 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.854991853, 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042143574000000003, 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1636 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:21.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224731+0000", 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5600000000000001e-06 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224738+0000", 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224756+0000", 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8153999999999999e-05 2024-09-14T07:08:21.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224770+0000", 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4168999999999999e-05 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266618+0000", 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041847662000000001 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266730+0000", 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000111993 2024-09-14T07:08:21.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266873+0000", 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014374400000000001 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1643 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 8978432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8539108180000001, 2024-09-14T07:08:21.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042103224000000002, 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1643 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225812+0000", 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.207e-06 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225818+0000", 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225836+0000", 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8221999999999999e-05 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225849+0000", 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.358e-05 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267705+0000", 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855723999999997 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267805+0000", 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9437000000000004e-05 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267914+0000", 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109554 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.119 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1644 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8538525899999998, 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042307838, 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1644 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:21.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225870+0000", 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2270000000000001e-06 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225876+0000", 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2569999999999998e-06 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225894+0000", 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8385000000000001e-05 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225936+0000", 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2159000000000001e-05 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267952+0000", 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04201585 2024-09-14T07:08:21.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268067+0000", 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114985 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268177+0000", 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109975 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1645 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8537642129999998, 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042484558999999998, 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1645 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225959+0000", 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225978+0000", 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9466e-05 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225998+0000", 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9315999999999999e-05 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226013+0000", 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:21.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268213+0000", 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200410000000001 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268324+0000", 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011123799999999999 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268442+0000", 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117889 2024-09-14T07:08:21.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1646 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8536878350000001, 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042595283999999997, 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1646 2024-09-14T07:08:21.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226035+0000", 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5060000000000001e-06 2024-09-14T07:08:21.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226042+0000", 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4370000000000004e-06 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226060+0000", 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8266000000000001e-05 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226074+0000", 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4029000000000001e-05 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042405023999999999 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268551+0000", 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1973999999999999e-05 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268629+0000", 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8047999999999995e-05 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1647 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12910592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8536298009999999, 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042800796000000002, 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1647 2024-09-14T07:08:21.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226093+0000", 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.46e-06 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226099+0000", 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3530000000000002e-06 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226118+0000", 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9055999999999998e-05 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226132+0000", 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268673+0000", 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042541272999999998 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268775+0000", 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102302 2024-09-14T07:08:21.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268893+0000", 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117539 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1648 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:21.137 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8535706599999999, 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042904468000000001, 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1648 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226152+0000", 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.234e-06 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226157+0000", 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2000000000000002e-06 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226175+0000", 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7059e-05 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226204+0000", 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9536e-05 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268938+0000", 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042733698000000001 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268992+0000", 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4693e-05 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269056+0000", 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3047999999999997e-05 2024-09-14T07:08:21.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1649 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8534964519999999, 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043088794999999999, 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1649 2024-09-14T07:08:21.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226227+0000", 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.33e-06 2024-09-14T07:08:21.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226232+0000", 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.733e-06 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226251+0000", 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8816000000000001e-05 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226264+0000", 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-05 2024-09-14T07:08:21.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269097+0000", 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0428324 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269197+0000", 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100419 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269314+0000", 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116897 2024-09-14T07:08:21.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1651 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8534376039999998, 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043199128000000003, 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1651 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226285+0000", 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.237e-06 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226291+0000", 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1789999999999999e-06 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226311+0000", 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0656999999999999e-05 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226327+0000", 2024-09-14T07:08:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5948999999999999e-05 2024-09-14T07:08:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269352+0000", 2024-09-14T07:08:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043024830999999999 2024-09-14T07:08:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269419+0000", 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7281999999999997e-05 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269483+0000", 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3992999999999995e-05 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1700 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 32768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8533742760000003, 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043417208999999998, 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1700 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.151 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226349+0000", 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3629999999999999e-06 2024-09-14T07:08:21.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226354+0000", 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226372+0000", 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7612999999999999e-05 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226386+0000", 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4146e-05 2024-09-14T07:08:21.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269525+0000", 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043139380999999997 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269631+0000", 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105574 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269765+0000", 2024-09-14T07:08:21.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133759 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1702 3.13 3:cfeabdd5:::rbd_data.11616dafc67f.000000000000000f:9 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8532858559999998, 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043560261000000003, 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1702 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226437+0000", 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.567e-06 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226457+0000", 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395999999999999e-05 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226477+0000", 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9998999999999999e-05 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226491+0000", 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4503e-05 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269835+0000", 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043343995000000003 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269914+0000", 2024-09-14T07:08:21.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9018999999999997e-05 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269996+0000", 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1781999999999997e-05 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1740 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8532089620000001, 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043673828999999997, 2024-09-14T07:08:21.159 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1740 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226514+0000", 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4869999999999999e-06 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226520+0000", 2024-09-14T07:08:21.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.976e-06 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226538+0000", 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7972999999999999e-05 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226551+0000", 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3142999999999999e-05 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270041+0000", 2024-09-14T07:08:21.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043490174999999999 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270101+0000", 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9515999999999997e-05 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270187+0000", 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5558999999999999e-05 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.7884347009999999, 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.164 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:08:21.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:08:21.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4803.0:1903 3.1f e43/21 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head v 43'21, mlcod=42'19)", 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.771273882, 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.057064131999999997, 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308449+0000", 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.279e-06 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324156+0000", 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015707199000000002 2024-09-14T07:08:21.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324195+0000", 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9070999999999999e-05 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324217+0000", 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1752000000000001e-05 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324333+0000", 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011607000000000001 2024-09-14T07:08:21.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324359+0000", 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6296e-05 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365477+0000", 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041117065000000001 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365512+0000", 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-05 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.173 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.174 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.174 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:21.175 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:21.176 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:21.173+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:21.276 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:21.273+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:21.357 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:21.358 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:21.358 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:21.358 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:21.371 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:21.416 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:21.413+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:21.479 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:21.478+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:21.541 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:21.541 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:21.541 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:21.541 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:21.541 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:21.555 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:21.579 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:21.578+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:21.597 INFO:teuthology.orchestra.run.smithi145.stdout:829 write 0x3ed7200 thru 0x3ee3ef4 (0xccf5 bytes) 2024-09-14T07:08:21.598 INFO:teuthology.orchestra.run.smithi145.stdout:834 read 0x325ee00 thru 0x3264be7 (0x5de8 bytes) 2024-09-14T07:08:21.600 INFO:teuthology.orchestra.run.smithi145.stdout:835 read 0x4c74a00 thru 0x4c76a9c (0x209d bytes) 2024-09-14T07:08:21.602 INFO:teuthology.orchestra.run.smithi145.stdout:836 punch from 0x56a3e00 to 0x56ab000, (0x7200 bytes) 2024-09-14T07:08:21.680 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:21.677+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:21.682 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:21.682 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:21.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1402 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216793+0000", 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4636252900000004, 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070034000000001, 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1402 2024-09-14T07:08:21.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216793+0000", 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216793+0000", 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216794+0000", 2024-09-14T07:08:21.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2759999999999999e-06 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216799+0000", 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2599999999999996e-06 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216819+0000", 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9775999999999998e-05 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216833+0000", 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260723+0000", 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043890167000000001 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260779+0000", 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5971000000000003e-05 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260863+0000", 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3770999999999995e-05 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1404 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.463565966, 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044358094000000001, 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.687 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1404 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216853+0000", 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216859+0000", 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2669999999999997e-06 2024-09-14T07:08:21.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216877+0000", 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7866000000000001e-05 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216891+0000", 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4712000000000001e-05 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260962+0000", 2024-09-14T07:08:21.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070703000000003 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261081+0000", 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011867 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261210+0000", 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012960599999999999 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1407 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4635082209999997, 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044521318999999997, 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1407 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216911+0000", 2024-09-14T07:08:21.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.333e-06 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216916+0000", 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0300000000000001e-06 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216933+0000", 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6616000000000002e-05 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216947+0000", 2024-09-14T07:08:21.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3923e-05 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261260+0000", 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044313105999999998 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261323+0000", 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2688999999999996e-05 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261431+0000", 2024-09-14T07:08:21.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108622 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1408 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.463452706, 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044764117999999999, 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1408 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216967+0000", 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4360000000000001e-06 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216987+0000", 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9689000000000001e-05 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217005+0000", 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8260000000000001e-05 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217018+0000", 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3396e-05 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261500+0000", 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044481494000000003 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261601+0000", 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101755 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261730+0000", 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128088 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1560 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221490+0000", 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4589281400000003, 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044131030000000002, 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1560 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221490+0000", 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221490+0000", 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221491+0000", 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221496+0000", 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2159999999999997e-06 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221515+0000", 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8844000000000001e-05 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221530+0000", 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4769e-05 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265391+0000", 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043861243000000001 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265498+0000", 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106464 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265621+0000", 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012340800000000001 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1561 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4588698190000002, 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044533453000000001, 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1561 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221569+0000", 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-05 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221589+0000", 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9266000000000002e-05 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221602+0000", 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.366e-05 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265854+0000", 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044251429000000002 2024-09-14T07:08:21.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265962+0000", 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266082+0000", 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011999500000000001 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1562 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4587938920000001, 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044744586000000003, 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1562 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.709 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221625+0000", 2024-09-14T07:08:21.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221631+0000", 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4729999999999999e-06 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221648+0000", 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7346e-05 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221663+0000", 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266141+0000", 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044478651000000001 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266249+0000", 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107371 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266369+0000", 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120258 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1563 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4587363169999996, 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044928442999999998, 2024-09-14T07:08:21.713 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1563 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221683+0000", 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.55e-06 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221689+0000", 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1699999999999996e-06 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221706+0000", 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7286e-05 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221719+0000", 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2959000000000001e-05 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266405+0000", 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044686664000000001 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266488+0000", 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2243999999999994e-05 2024-09-14T07:08:21.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266610+0000", 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012256999999999999 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1698 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224160+0000", 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4562580040000004, 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044028734999999999, 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.718 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1698 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224160+0000", 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224160+0000", 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224161+0000", 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.313e-06 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224167+0000", 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.147e-06 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224184+0000", 2024-09-14T07:08:21.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7520000000000002e-05 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224198+0000", 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4141999999999999e-05 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268059+0000", 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043860540000000003 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268117+0000", 2024-09-14T07:08:21.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8063999999999998e-05 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268189+0000", 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2008999999999994e-05 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1699 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4562014860000003, 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044262176, 2024-09-14T07:08:21.722 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1699 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224218+0000", 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224254+0000", 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6525e-05 2024-09-14T07:08:21.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224274+0000", 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0049999999999999e-05 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224289+0000", 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268246+0000", 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043957152999999999 2024-09-14T07:08:21.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268354+0000", 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107433 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012504200000000001 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1701 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4561055359999999, 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044338850999999999, 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1701 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224314+0000", 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224320+0000", 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224339+0000", 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8592000000000001e-05 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224353+0000", 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.432e-05 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268522+0000", 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044169491999999998 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268578+0000", 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5099999999999998e-05 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268651+0000", 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-05 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.730 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1703 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4560418540000004, 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044549686999999998, 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1703 2024-09-14T07:08:21.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224378+0000", 2024-09-14T07:08:21.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.573e-06 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224384+0000", 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224403+0000", 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9113000000000001e-05 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224417+0000", 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4049000000000001e-05 2024-09-14T07:08:21.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268700+0000", 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044282555000000001 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268805+0000", 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104858 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268926+0000", 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121296 2024-09-14T07:08:21.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1704 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9764864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4559805130000001, 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044649934000000002, 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:08:21.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224439+0000", 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224902+0000", 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046329699999999999 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224924+0000", 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2059e-05 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224941+0000", 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835999999999999e-05 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268962+0000", 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044020653 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269012+0000", 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0312000000000002e-05 2024-09-14T07:08:21.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269088+0000", 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5457000000000006e-05 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1705 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.455448079, 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044393046999999998, 2024-09-14T07:08:21.739 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224972+0000", 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.784e-06 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224979+0000", 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224996+0000", 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225010+0000", 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3926e-05 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269132+0000", 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044121736000000002 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269242+0000", 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010979700000000001 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269363+0000", 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121155 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1706 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.743 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4553889169999996, 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044499060999999999, 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:21.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225030+0000", 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225036+0000", 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4530000000000001e-06 2024-09-14T07:08:21.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225053+0000", 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6983e-05 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225066+0000", 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269399+0000", 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044332900000000001 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269449+0000", 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9960000000000003e-05 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269528+0000", 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9221999999999995e-05 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1707 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:21.747 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4553343459999999, 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044705944999999997, 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225085+0000", 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225090+0000", 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.113e-06 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225107+0000", 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6569999999999999e-05 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225135+0000", 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8524999999999999e-05 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269571+0000", 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044435781000000001 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269676+0000", 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105374 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269790+0000", 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113415 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1708 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4552630850000003, 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044874641, 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.752 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225156+0000", 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225162+0000", 2024-09-14T07:08:21.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5960000000000003e-06 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225179+0000", 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7235999999999999e-05 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225192+0000", 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269839+0000", 2024-09-14T07:08:21.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044646439000000003 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269926+0000", 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7303000000000001e-05 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270030+0000", 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001036 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1709 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.4552069669999996, 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045094769, 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225213+0000", 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6339999999999999e-06 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225218+0000", 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3959999999999997e-06 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225236+0000", 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8053000000000001e-05 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225250+0000", 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3583e-05 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270081+0000", 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044830868000000003 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270179+0000", 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8645000000000002e-05 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270306+0000", 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012658999999999999 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.760 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1710 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.455149392, 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045251099000000003, 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.761 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225270+0000", 2024-09-14T07:08:21.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.173e-06 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225276+0000", 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6840000000000001e-06 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225292+0000", 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6722e-05 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225306+0000", 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3456e-05 2024-09-14T07:08:21.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270348+0000", 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045042722 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270431+0000", 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2182999999999996e-05 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270520+0000", 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9159000000000005e-05 2024-09-14T07:08:21.764 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3835489189999999, 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:21.765 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:08:21.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:08:21.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:08:21.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:08:21.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:21.770 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:21.772 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:21.781 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:21.778+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:21.882 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:21.877+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:21.938 INFO:teuthology.orchestra.run.smithi145.stdout:837 punch from 0x247ec00 to 0x2480a00, (0x1e00 bytes) 2024-09-14T07:08:21.957 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:21.957 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:21.957 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:21.957 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:21.971 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:22.015 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:22.013+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:22.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:08:22.084 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:22.081+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:22.109 INFO:teuthology.orchestra.run.smithi145.stdout:838 read 0xa103a00 thru 0xa110ba1 (0xd1a2 bytes) 2024-09-14T07:08:22.111 INFO:teuthology.orchestra.run.smithi145.stdout:839 write 0x9f76200 thru 0x9f7b3c5 (0x51c6 bytes) 2024-09-14T07:08:22.112 INFO:teuthology.orchestra.run.smithi145.stdout:843 write 0x9ac6000 thru 0x9ad2a01 (0xca02 bytes) 2024-09-14T07:08:22.113 INFO:teuthology.orchestra.run.smithi145.stdout:844 write 0x9292a00 thru 0x929d247 (0xa848 bytes) 2024-09-14T07:08:22.115 INFO:teuthology.orchestra.run.smithi145.stdout:846 write 0x54d000 thru 0x55777e (0xa77f bytes) 2024-09-14T07:08:22.116 INFO:teuthology.orchestra.run.smithi145.stdout:851 punch from 0x2b97000 to 0x2b9c200, (0x5200 bytes) 2024-09-14T07:08:22.142 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:22.142 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:22.142 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:22.142 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:22.142 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:22.160 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:22.203 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:22.201+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:22.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:22.253 INFO:teuthology.orchestra.run.smithi145.stdout:852 punch from 0xcbea000 to 0xcbf9a00, (0xfa00 bytes) 2024-09-14T07:08:22.264 INFO:teuthology.orchestra.run.smithi145.stdout:853 punch from 0xa5cd600 to 0xa5dd400, (0xfe00 bytes) 2024-09-14T07:08:22.274 INFO:teuthology.orchestra.run.smithi145.stdout:854 read 0x1178600 thru 0x117a5af (0x1fb0 bytes) 2024-09-14T07:08:22.275 INFO:teuthology.orchestra.run.smithi145.stdout:855 write 0x9a38c00 thru 0x9a41b15 (0x8f16 bytes) 2024-09-14T07:08:22.285 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:22.282+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:22.321 INFO:teuthology.orchestra.run.smithi145.stdout:858 write 0xee30000 thru 0xee35be9 (0x5bea bytes) 2024-09-14T07:08:22.322 INFO:teuthology.orchestra.run.smithi145.stdout:860 punch from 0x1944400 to 0x194b400, (0x7000 bytes) 2024-09-14T07:08:22.323 INFO:teuthology.orchestra.run.smithi145.stdout:861 read 0x2b2c800 thru 0x2b2f6cb (0x2ecc bytes) 2024-09-14T07:08:22.327 INFO:teuthology.orchestra.run.smithi145.stdout:862 read 0xd8ef200 thru 0xd8f08e9 (0x16ea bytes) 2024-09-14T07:08:22.328 INFO:teuthology.orchestra.run.smithi145.stdout:863 read 0xe3cae00 thru 0xe3d21fb (0x73fc bytes) 2024-09-14T07:08:22.337 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:22.337 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.505982014000001, 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:22.338 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:22.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:22.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:22.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:22.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.505207666, 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:22.343 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:22.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:22.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:22.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3038 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2069854919999994, 2024-09-14T07:08:22.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046439165999999997, 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3038 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129535+0000", 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3230000000000001e-06 2024-09-14T07:08:22.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129541+0000", 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.054e-06 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129559+0000", 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8329e-05 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129573+0000", 2024-09-14T07:08:22.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175474+0000", 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045901019000000001 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175554+0000", 2024-09-14T07:08:22.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0661000000000006e-05 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175973+0000", 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041842300000000002 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3039 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.206927812, 2024-09-14T07:08:22.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046967035999999997, 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3039 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129592+0000", 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.206e-06 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129598+0000", 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2340000000000002e-06 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129616+0000", 2024-09-14T07:08:22.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8003e-05 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129629+0000", 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2913e-05 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176018+0000", 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046389224999999999 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176108+0000", 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176558+0000", 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044997100000000002 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.358 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3040 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2068386100000001, 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047451337000000003, 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3040 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:22.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129682+0000", 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.607e-06 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129688+0000", 2024-09-14T07:08:22.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129708+0000", 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0047e-05 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129723+0000", 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.502e-05 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176597+0000", 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046873259 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176684+0000", 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7349999999999995e-05 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177132+0000", 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044784099999999998 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3041 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2067753069999991, 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047622767000000003, 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3041 2024-09-14T07:08:22.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129745+0000", 2024-09-14T07:08:22.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129751+0000", 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129773+0000", 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782999999999999e-05 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129788+0000", 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4606999999999999e-05 2024-09-14T07:08:22.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177167+0000", 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047378941000000001 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177253+0000", 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5704999999999993e-05 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177367+0000", 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113861 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3042 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2067118679999993, 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048188976000000001, 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.367 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3042 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129808+0000", 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129814+0000", 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7699999999999998e-06 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129832+0000", 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.751e-05 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129844+0000", 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2797e-05 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177436+0000", 2024-09-14T07:08:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047591574999999997 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177545+0000", 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109446 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177996+0000", 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045071800000000002 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3043 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2066554140000001, 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048376965000000001, 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3043 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129865+0000", 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129870+0000", 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129888+0000", 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.755e-05 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129902+0000", 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178049+0000", 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048146648 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178131+0000", 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2619000000000003e-05 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178241+0000", 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109262 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3044 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2065833349999995, 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048575449, 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3044 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129937+0000", 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1930000000000001e-06 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129943+0000", 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129961+0000", 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8034e-05 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129975+0000", 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3696000000000001e-05 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178293+0000", 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048318051000000001 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178392+0000", 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9038999999999998e-05 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178511+0000", 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119493 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3045 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2065240789999994, 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049079214000000003, 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3045 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129996+0000", 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1170000000000001e-06 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130002+0000", 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130024+0000", 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1763000000000002e-05 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130059+0000", 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.536e-05 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178555+0000", 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048495599 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178636+0000", 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1112000000000005e-05 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179074+0000", 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043820000000000003 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3046 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2064374030000007, 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049550174000000002, 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3046 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130083+0000", 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130089+0000", 2024-09-14T07:08:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130108+0000", 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9046999999999999e-05 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130122+0000", 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4137e-05 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179126+0000", 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049003927000000003 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179221+0000", 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4717999999999997e-05 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179632+0000", 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041092199999999998 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3047 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2063767060000004, 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050053944000000003, 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3047 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130144+0000", 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3030000000000001e-06 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130149+0000", 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6230000000000002e-06 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130169+0000", 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9386999999999999e-05 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130182+0000", 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3706000000000001e-05 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179677+0000", 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049494109000000001 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179763+0000", 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6473999999999999e-05 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180196+0000", 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043334200000000003 2024-09-14T07:08:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3048 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2063167870000004, 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050582655999999997, 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3048 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130204+0000", 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.266e-06 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130219+0000", 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5549999999999999e-05 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130235+0000", 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6030000000000001e-05 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130249+0000", 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3977e-05 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180264+0000", 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050014851999999999 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180359+0000", 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5400000000000001e-05 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180785+0000", 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042558100000000002 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3049 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2062493209999996, 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.051156504999999998, 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3049 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130271+0000", 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130277+0000", 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6370000000000004e-06 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130297+0000", 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.997e-05 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130311+0000", 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4358999999999999e-05 2024-09-14T07:08:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180833+0000", 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050521863 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180936+0000", 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010296799999999999 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.181426+0000", 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049044799999999999 2024-09-14T07:08:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3149 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.191062745, 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046930569999999998, 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3149 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145458+0000", 2024-09-14T07:08:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145463+0000", 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6169999999999998e-06 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145486+0000", 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2626e-05 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145500+0000", 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4243000000000001e-05 2024-09-14T07:08:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191669+0000", 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046169090000000003 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191811+0000", 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014187100000000001 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192387+0000", 2024-09-14T07:08:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057567299999999996 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3150 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.190996385, 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047680769999999997, 2024-09-14T07:08:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3150 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145524+0000", 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.39e-06 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145647+0000", 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123242 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145673+0000", 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145711+0000", 2024-09-14T07:08:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8745999999999998e-05 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192444+0000", 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046732462000000002 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192602+0000", 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015825400000000001 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193203+0000", 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00060150600000000002 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3172 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.1906105149999995, 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048262865000000002, 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3172 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145911+0000", 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145920+0000", 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2969999999999992e-06 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145947+0000", 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7149999999999999e-05 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145965+0000", 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8295999999999999e-05 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193303+0000", 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047338247 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193492+0000", 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018843500000000001 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194171+0000", 2024-09-14T07:08:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067951700000000003 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3173 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.1905251589999999, 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048937952999999999, 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3173 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145995+0000", 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146021+0000", 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5684e-05 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146044+0000", 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2503000000000001e-05 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146059+0000", 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5043e-05 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194258+0000", 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04819975 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194372+0000", 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113373 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194932+0000", 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056004399999999999 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.494024668, 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:08:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.197096462, 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:08:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:22.432 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:22.433 INFO:teuthology.orchestra.run.smithi145.stdout:864 trunc from 0xee35c00 to 0x304fe00 2024-09-14T07:08:22.433 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:22.433 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:22.434 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:22.434 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:22.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:22.435 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:22.435 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:22.435 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:22.430+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:22.443 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:22.443 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:22.464 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:22.485 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:22.520 INFO:teuthology.orchestra.run.smithi145.stdout:868 punch from 0x189fc00 to 0x18a0400, (0x800 bytes) 2024-09-14T07:08:22.523 INFO:teuthology.orchestra.run.smithi145.stdout:870 punch from 0x2a50e00 to 0x2a58a00, (0x7c00 bytes) 2024-09-14T07:08:22.523 INFO:teuthology.orchestra.run.smithi145.stdout:871 read 0x14fbe00 thru 0x150578d (0x998e bytes) 2024-09-14T07:08:22.533 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:22.529+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:22.553 INFO:teuthology.orchestra.run.smithi145.stdout:872 write 0x43b2e00 thru 0x43bd4e3 (0xa6e4 bytes) 2024-09-14T07:08:22.556 INFO:teuthology.orchestra.run.smithi145.stdout:873 punch from 0x3647200 to 0x364b400, (0x4200 bytes) 2024-09-14T07:08:22.557 INFO:teuthology.orchestra.run.smithi145.stdout:875 read 0x3931000 thru 0x3937a07 (0x6a08 bytes) 2024-09-14T07:08:22.558 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:22.558 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:22.559 INFO:teuthology.orchestra.run.smithi145.stdout:880 punch from 0x28f2c00 to 0x2901e00, (0xf200 bytes) 2024-09-14T07:08:22.559 INFO:teuthology.orchestra.run.smithi145.stdout:881 read 0x977000 thru 0x97d49d (0x649e bytes) 2024-09-14T07:08:22.560 INFO:teuthology.orchestra.run.smithi145.stdout:882 punch from 0xfe8000 to 0xff5600, (0xd600 bytes) 2024-09-14T07:08:22.560 INFO:teuthology.orchestra.run.smithi145.stdout:883 read 0x267a400 thru 0x267b618 (0x1219 bytes) 2024-09-14T07:08:22.564 INFO:teuthology.orchestra.run.smithi145.stdout:884 trunc from 0x43bd600 to 0x4074c00 2024-09-14T07:08:22.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:22.587 INFO:teuthology.orchestra.run.smithi145.stdout:885 read 0x37a5800 thru 0x37a9a96 (0x4297 bytes) 2024-09-14T07:08:22.590 INFO:teuthology.orchestra.run.smithi145.stdout:886 read 0x186e000 thru 0x1872685 (0x4686 bytes) 2024-09-14T07:08:22.627 INFO:teuthology.orchestra.run.smithi145.stdout:887 write 0xb6a3000 thru 0xb6b08db (0xd8dc bytes) 2024-09-14T07:08:22.629 INFO:teuthology.orchestra.run.smithi145.stdout:888 read 0x8e91e00 thru 0x8e95d94 (0x3f95 bytes) 2024-09-14T07:08:22.632 INFO:teuthology.orchestra.run.smithi145.stdout:890 write 0x2076e00 thru 0x2086dd3 (0xffd4 bytes) 2024-09-14T07:08:22.634 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:22.629+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:22.680 INFO:teuthology.orchestra.run.smithi145.stdout:891 trunc from 0xb6b0a00 to 0x7ef000 2024-09-14T07:08:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:22.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:22.735 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:22.733+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:22.789 INFO:teuthology.orchestra.run.smithi145.stdout:892 write 0x1e4f000 thru 0x1e5803d (0x903e bytes) 2024-09-14T07:08:22.791 INFO:teuthology.orchestra.run.smithi145.stdout:893 punch from 0x157da00 to 0x1584200, (0x6800 bytes) 2024-09-14T07:08:22.791 INFO:teuthology.orchestra.run.smithi145.stdout:894 read 0x15f3e00 thru 0x15fde70 (0xa071 bytes) 2024-09-14T07:08:22.796 INFO:teuthology.orchestra.run.smithi145.stdout:895 read 0x2e9000 thru 0x2f0876 (0x7877 bytes) 2024-09-14T07:08:22.797 INFO:teuthology.orchestra.run.smithi145.stdout:896 read 0x1083600 thru 0x109115b (0xdb5c bytes) 2024-09-14T07:08:22.814 INFO:teuthology.orchestra.run.smithi145.stdout:897 trunc from 0x1e58200 to 0xc80e000 2024-09-14T07:08:22.833 INFO:teuthology.orchestra.run.smithi145.stdout:899 write 0xc09ce00 thru 0xc0ab032 (0xe233 bytes) 2024-09-14T07:08:22.834 INFO:teuthology.orchestra.run.smithi145.stdout:900 write 0x6272e00 thru 0x6281160 (0xe361 bytes) 2024-09-14T07:08:22.834 INFO:teuthology.orchestra.run.smithi145.stdout:902 write 0xa807600 thru 0xa80df95 (0x6996 bytes) 2024-09-14T07:08:22.835 INFO:teuthology.orchestra.run.smithi145.stdout:906 write 0x6d5cc00 thru 0x6d62e4d (0x624e bytes) 2024-09-14T07:08:22.836 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:22.834+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:22.852 INFO:teuthology.orchestra.run.smithi145.stdout:907 trunc from 0xc80e000 to 0x275c600 2024-09-14T07:08:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:22.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:22.917 INFO:teuthology.orchestra.run.smithi145.stdout:908 trunc from 0x275c600 to 0x86c4400 2024-09-14T07:08:22.937 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:22.933+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:22.966 INFO:teuthology.orchestra.run.smithi145.stdout:909 write 0xbeeb800 thru 0xbef3d97 (0x8598 bytes) 2024-09-14T07:08:22.993 INFO:teuthology.orchestra.run.smithi145.stdout:910 trunc from 0xbef3e00 to 0x1826c00 2024-09-14T07:08:23.014 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:23.015 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:23.015 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 3, 4, 5, 2] 2024-09-14T07:08:23.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-14T07:08:23.028 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:08:23.038 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:23.033+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:23.058 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:23.058 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:23.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:23.167 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:23.166+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:23.240 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:23.237+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:23.244 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:23.244 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:23.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:23.297 INFO:teuthology.orchestra.run.smithi145.stdout:911 punch from 0xf4fe00 to 0xf5de00, (0xe000 bytes) 2024-09-14T07:08:23.300 INFO:teuthology.orchestra.run.smithi145.stdout:912 trunc from 0x1826c00 to 0x26bd000 2024-09-14T07:08:23.324 INFO:teuthology.orchestra.run.smithi145.stdout:913 read 0x1569a00 thru 0x15783c2 (0xe9c3 bytes) 2024-09-14T07:08:23.329 INFO:teuthology.orchestra.run.smithi145.stdout:914 read 0x463c00 thru 0x4665a8 (0x29a9 bytes) 2024-09-14T07:08:23.331 INFO:teuthology.orchestra.run.smithi145.stdout:917 read 0x1f74200 thru 0x1f7f227 (0xb028 bytes) 2024-09-14T07:08:23.340 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:08:23.338+0000 7f1e6b537640 -1 mon.a@0(leader).osd e47 definitely_dead 0 2024-09-14T07:08:23.340 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:23.337+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:23.358 INFO:teuthology.orchestra.run.smithi145.stdout:918 trunc from 0x26bd000 to 0xc956e00 2024-09-14T07:08:23.409 INFO:teuthology.orchestra.run.smithi145.stdout:925 trunc from 0xc956e00 to 0x5cc000 2024-09-14T07:08:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:23.441 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:23.437+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:23.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:23.503 INFO:teuthology.orchestra.run.smithi145.stdout:926 write 0x48db400 thru 0x48e75aa (0xc1ab bytes) 2024-09-14T07:08:23.505 INFO:teuthology.orchestra.run.smithi145.stdout:928 read 0xab8800 thru 0xac46d9 (0xbeda bytes) 2024-09-14T07:08:23.511 INFO:teuthology.orchestra.run.smithi145.stdout:929 trunc from 0x48e7600 to 0x266d000 2024-09-14T07:08:23.541 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:23.538+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:23.542 INFO:teuthology.orchestra.run.smithi145.stdout:931 read 0xb02800 thru 0xb0a6d9 (0x7eda bytes) 2024-09-14T07:08:23.543 INFO:teuthology.orchestra.run.smithi145.stdout:933 read 0x25ab200 thru 0x25b34fb (0x82fc bytes) 2024-09-14T07:08:23.568 INFO:teuthology.orchestra.run.smithi145.stdout:934 write 0x444fc00 thru 0x44543c7 (0x47c8 bytes) 2024-09-14T07:08:23.577 INFO:teuthology.orchestra.run.smithi145.stdout:935 trunc from 0x4454400 to 0x1887a00 2024-09-14T07:08:23.610 INFO:teuthology.orchestra.run.smithi145.stdout:936 read 0x2b2a00 thru 0x2b59b0 (0x2fb1 bytes) 2024-09-14T07:08:23.630 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:23.631 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:23.642 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:23.638+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:23.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:23.655 INFO:teuthology.orchestra.run.smithi145.stdout:939 write 0xbf36a00 thru 0xbf439b7 (0xcfb8 bytes) 2024-09-14T07:08:23.657 INFO:teuthology.orchestra.run.smithi145.stdout:941 write 0x463c000 thru 0x4646a85 (0xaa86 bytes) 2024-09-14T07:08:23.657 INFO:teuthology.orchestra.run.smithi145.stdout:944 read 0x1e91400 thru 0x1e9f3de (0xdfdf bytes) 2024-09-14T07:08:23.688 INFO:teuthology.orchestra.run.smithi145.stdout:946 write 0xbf57a00 thru 0xbf5d6a9 (0x5caa bytes) 2024-09-14T07:08:23.690 INFO:teuthology.orchestra.run.smithi145.stdout:947 read 0x87f1200 thru 0x87f2404 (0x1205 bytes) 2024-09-14T07:08:23.722 INFO:teuthology.orchestra.run.smithi145.stdout:948 write 0xe25a000 thru 0xe26627e (0xc27f bytes) 2024-09-14T07:08:23.724 INFO:teuthology.orchestra.run.smithi145.stdout:949 read 0xce24a00 thru 0xce270b6 (0x26b7 bytes) 2024-09-14T07:08:23.743 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:23.741+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:23.755 INFO:teuthology.orchestra.run.smithi145.stdout:950 trunc from 0xe266400 to 0x2906400 2024-09-14T07:08:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:23.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:23.854 INFO:teuthology.orchestra.run.smithi145.stdout:952 write 0x84e4000 thru 0x84e50b9 (0x10ba bytes) 2024-09-14T07:08:23.857 INFO:teuthology.orchestra.run.smithi145.stdout:953 write 0x631fa00 thru 0x6328639 (0x8c3a bytes) 2024-09-14T07:08:23.857 INFO:teuthology.orchestra.run.smithi145.stdout:954 punch from 0x1187600 to 0x1195c00, (0xe600 bytes) 2024-09-14T07:08:23.858 INFO:teuthology.orchestra.run.smithi145.stdout:955 read 0x475f600 thru 0x4764e54 (0x5855 bytes) 2024-09-14T07:08:23.859 INFO:teuthology.orchestra.run.smithi145.stdout:957 read 0x6708600 thru 0x670fe4f (0x7850 bytes) 2024-09-14T07:08:23.860 INFO:teuthology.orchestra.run.smithi145.stdout:958 read 0x5f07000 thru 0x5f0f1e3 (0x81e4 bytes) 2024-09-14T07:08:23.861 INFO:teuthology.orchestra.run.smithi145.stdout:959 trunc from 0x84e5200 to 0x83fc800 2024-09-14T07:08:23.879 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:23.878+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:23.893 INFO:teuthology.orchestra.run.smithi145.stdout:960 write 0x564ac00 thru 0x56523e1 (0x77e2 bytes) 2024-09-14T07:08:23.896 INFO:teuthology.orchestra.run.smithi145.stdout:962 punch from 0x2c52600 to 0x2c53400, (0xe00 bytes) 2024-09-14T07:08:23.932 INFO:teuthology.orchestra.run.smithi145.stdout:965 write 0xe487600 thru 0xe493782 (0xc183 bytes) 2024-09-14T07:08:23.935 INFO:teuthology.orchestra.run.smithi145.stdout:967 read 0xf0b600 thru 0xf11dda (0x67db bytes) 2024-09-14T07:08:23.936 INFO:teuthology.orchestra.run.smithi145.stdout:968 write 0xcc88200 thru 0xcc96980 (0xe781 bytes) 2024-09-14T07:08:23.937 INFO:teuthology.orchestra.run.smithi145.stdout:969 read 0xded7a00 thru 0xdedd862 (0x5e63 bytes) 2024-09-14T07:08:23.937 INFO:teuthology.orchestra.run.smithi145.stdout:972 read 0x6a55000 thru 0x6a6299f (0xd9a0 bytes) 2024-09-14T07:08:23.938 INFO:teuthology.orchestra.run.smithi145.stdout:973 punch from 0xb2f1c00 to 0xb300600, (0xea00 bytes) 2024-09-14T07:08:23.939 INFO:teuthology.orchestra.run.smithi145.stdout:975 punch from 0x3878e00 to 0x3880e00, (0x8000 bytes) 2024-09-14T07:08:23.940 INFO:teuthology.orchestra.run.smithi145.stdout:976 write 0x2e1fc00 thru 0x2e26fc0 (0x73c1 bytes) 2024-09-14T07:08:23.940 INFO:teuthology.orchestra.run.smithi145.stdout:978 read 0x29f6000 thru 0x29fd0e7 (0x70e8 bytes) 2024-09-14T07:08:23.943 INFO:teuthology.orchestra.run.smithi145.stdout:983 write 0x8f00a00 thru 0x8f03914 (0x2f15 bytes) 2024-09-14T07:08:23.944 INFO:teuthology.orchestra.run.smithi145.stdout:985 write 0xcd63600 thru 0xcd70f9c (0xd99d bytes) 2024-09-14T07:08:23.944 INFO:teuthology.orchestra.run.smithi145.stdout:989 write 0x5c9e600 thru 0x5cad0e0 (0xeae1 bytes) 2024-09-14T07:08:23.944 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:23.941+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:23.946 INFO:teuthology.orchestra.run.smithi145.stdout:992 write 0x100200 thru 0x10c819 (0xc61a bytes) 2024-09-14T07:08:23.947 INFO:teuthology.orchestra.run.smithi145.stdout:993 punch from 0x8049a00 to 0x804f800, (0x5e00 bytes) 2024-09-14T07:08:23.949 INFO:teuthology.orchestra.run.smithi145.stdout:994 read 0xa703c00 thru 0xa704b7d (0xf7e bytes) 2024-09-14T07:08:23.959 INFO:teuthology.orchestra.run.smithi145.stdout:997 write 0xd9c3e00 thru 0xd9c5738 (0x1939 bytes) 2024-09-14T07:08:23.963 INFO:teuthology.orchestra.run.smithi145.stdout:998 punch from 0x7d64200 to 0x7d68200, (0x4000 bytes) 2024-09-14T07:08:23.963 INFO:teuthology.orchestra.run.smithi145.stdout:999 punch from 0x95b9200 to 0x95c6e00, (0xdc00 bytes) 2024-09-14T07:08:23.963 INFO:teuthology.orchestra.run.smithi145.stdout:1002 read 0xa075a00 thru 0xa07d46d (0x7a6e bytes) 2024-09-14T07:08:23.964 INFO:teuthology.orchestra.run.smithi145.stdout:1004 read 0x3c00000 thru 0x3c0f0fd (0xf0fe bytes) 2024-09-14T07:08:23.964 INFO:teuthology.orchestra.run.smithi145.stdout:1005 punch from 0x27e2000 to 0x27ea200, (0x8200 bytes) 2024-09-14T07:08:23.978 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:23.978 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:23.987 INFO:teuthology.orchestra.run.smithi145.stdout:1006 trunc from 0xe493800 to 0x2bbb400 2024-09-14T07:08:23.993 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:24.030 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.4. 2024-09-14T07:08:24.045 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:24.041+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:24.129 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:24.129 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:24.143 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:24.187 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:24.185+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:24.247 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:24.246+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:24.309 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:24.309 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:24.323 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:24.348 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:24.346+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:24.444 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:24.444 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:24.449 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:24.446+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:24.460 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:24.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:24.551 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:24.550+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:24.577 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:24.577 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:24.591 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:24.650 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:24.650+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:24.661 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:24.678 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:24.694 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:24.695 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:24.710 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:24.710 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:24.714 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:24.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:24.735 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:24.754 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:24.771 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:24.769+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:24.851 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1402 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216793+0000", 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.632432573, 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070034000000001, 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1402 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216793+0000", 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216793+0000", 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216794+0000", 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2759999999999999e-06 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216799+0000", 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2599999999999996e-06 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216819+0000", 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9775999999999998e-05 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216833+0000", 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260723+0000", 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043890167000000001 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260779+0000", 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5971000000000003e-05 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260863+0000", 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3770999999999995e-05 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.856 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1404 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6323732490000005, 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044358094000000001, 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.857 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1404 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216853+0000", 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216859+0000", 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2669999999999997e-06 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216877+0000", 2024-09-14T07:08:24.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7866000000000001e-05 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216891+0000", 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4712000000000001e-05 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260962+0000", 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070703000000003 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261081+0000", 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011867 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261210+0000", 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012960599999999999 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1407 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.861 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6323155040000001, 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044521318999999997, 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1407 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216911+0000", 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.333e-06 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216916+0000", 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0300000000000001e-06 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216933+0000", 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6616000000000002e-05 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216947+0000", 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3923e-05 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261260+0000", 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044313105999999998 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261323+0000", 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2688999999999996e-05 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261431+0000", 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108622 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.865 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1408 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6322599889999996, 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044764117999999999, 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1408 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216967+0000", 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4360000000000001e-06 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216987+0000", 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9689000000000001e-05 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217005+0000", 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8260000000000001e-05 2024-09-14T07:08:24.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217018+0000", 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3396e-05 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261500+0000", 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044481494000000003 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261601+0000", 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101755 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261730+0000", 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128088 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1560 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13631488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221490+0000", 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6277354229999998, 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044131030000000002, 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1560 2024-09-14T07:08:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221490+0000", 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221490+0000", 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221491+0000", 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221496+0000", 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2159999999999997e-06 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221515+0000", 2024-09-14T07:08:24.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8844000000000001e-05 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221530+0000", 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4769e-05 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265391+0000", 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043861243000000001 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265498+0000", 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106464 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265621+0000", 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012340800000000001 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1561 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:24.875 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6276771019999998, 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044533453000000001, 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1561 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:08:24.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221569+0000", 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-05 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221589+0000", 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9266000000000002e-05 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221602+0000", 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.366e-05 2024-09-14T07:08:24.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265854+0000", 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044251429000000002 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265962+0000", 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266082+0000", 2024-09-14T07:08:24.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011999500000000001 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1562 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6276011749999997, 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044744586000000003, 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1562 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:24.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221625+0000", 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221631+0000", 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4729999999999999e-06 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221648+0000", 2024-09-14T07:08:24.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7346e-05 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221663+0000", 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266141+0000", 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044478651000000001 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266249+0000", 2024-09-14T07:08:24.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107371 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266369+0000", 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120258 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1563 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6275436000000001, 2024-09-14T07:08:24.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044928442999999998, 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1563 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221683+0000", 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.55e-06 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221689+0000", 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1699999999999996e-06 2024-09-14T07:08:24.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221706+0000", 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7286e-05 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221719+0000", 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2959000000000001e-05 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266405+0000", 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044686664000000001 2024-09-14T07:08:24.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266488+0000", 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2243999999999994e-05 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266610+0000", 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012256999999999999 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.888 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1698 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 5832704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224160+0000", 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.625065287, 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044028734999999999, 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.890 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1698 2024-09-14T07:08:24.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.890 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224160+0000", 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224160+0000", 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224161+0000", 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.313e-06 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224167+0000", 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.147e-06 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224184+0000", 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7520000000000002e-05 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224198+0000", 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4141999999999999e-05 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268059+0000", 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043860540000000003 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268117+0000", 2024-09-14T07:08:24.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8063999999999998e-05 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268189+0000", 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2008999999999994e-05 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.894 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1699 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6250087689999999, 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044262176, 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1699 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224218+0000", 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:08:24.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224254+0000", 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6525e-05 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224274+0000", 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0049999999999999e-05 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224289+0000", 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268246+0000", 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043957152999999999 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268354+0000", 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107433 2024-09-14T07:08:24.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012504200000000001 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1701 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:24.899 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6249128190000004, 2024-09-14T07:08:24.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044338850999999999, 2024-09-14T07:08:24.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1701 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224314+0000", 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224320+0000", 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224339+0000", 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8592000000000001e-05 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224353+0000", 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.432e-05 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268522+0000", 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044169491999999998 2024-09-14T07:08:24.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268578+0000", 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5099999999999998e-05 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268651+0000", 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-05 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1703 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.624849137, 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044549686999999998, 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1703 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:24.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224378+0000", 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.573e-06 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224384+0000", 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224403+0000", 2024-09-14T07:08:24.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9113000000000001e-05 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224417+0000", 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4049000000000001e-05 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268700+0000", 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044282555000000001 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268805+0000", 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104858 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268926+0000", 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121296 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1704 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9764864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6247877959999997, 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044649934000000002, 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224439+0000", 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224902+0000", 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046329699999999999 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224924+0000", 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2059e-05 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224941+0000", 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835999999999999e-05 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268962+0000", 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044020653 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269012+0000", 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0312000000000002e-05 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269088+0000", 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5457000000000006e-05 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1705 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6242553619999995, 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044393046999999998, 2024-09-14T07:08:24.914 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224972+0000", 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.784e-06 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224979+0000", 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224996+0000", 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225010+0000", 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3926e-05 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269132+0000", 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044121736000000002 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269242+0000", 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010979700000000001 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269363+0000", 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121155 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1706 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6241962000000001, 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044499060999999999, 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225030+0000", 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225036+0000", 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4530000000000001e-06 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225053+0000", 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6983e-05 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225066+0000", 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269399+0000", 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044332900000000001 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269449+0000", 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9960000000000003e-05 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269528+0000", 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9221999999999995e-05 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1707 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6241416290000004, 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044705944999999997, 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.923 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225085+0000", 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225090+0000", 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.113e-06 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225107+0000", 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6569999999999999e-05 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225135+0000", 2024-09-14T07:08:24.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8524999999999999e-05 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269571+0000", 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044435781000000001 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269676+0000", 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105374 2024-09-14T07:08:24.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269790+0000", 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113415 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1708 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6240703679999999, 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044874641, 2024-09-14T07:08:24.928 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:24.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:24.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225156+0000", 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225162+0000", 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5960000000000003e-06 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225179+0000", 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7235999999999999e-05 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225192+0000", 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269839+0000", 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044646439000000003 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269926+0000", 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7303000000000001e-05 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270030+0000", 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001036 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.933 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1709 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6240142500000001, 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045094769, 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225213+0000", 2024-09-14T07:08:24.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6339999999999999e-06 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225218+0000", 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3959999999999997e-06 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225236+0000", 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8053000000000001e-05 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225250+0000", 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3583e-05 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270081+0000", 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044830868000000003 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270179+0000", 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8645000000000002e-05 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270306+0000", 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012658999999999999 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1710 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6239566749999996, 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045251099000000003, 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.938 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225270+0000", 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.173e-06 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225276+0000", 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6840000000000001e-06 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225292+0000", 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6722e-05 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225306+0000", 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3456e-05 2024-09-14T07:08:24.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270348+0000", 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045042722 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270431+0000", 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2182999999999996e-05 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270520+0000", 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9159000000000005e-05 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.5523562020000004, 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:24.943 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:08:24.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:08:24.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:08:24.949 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.949 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.949 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.949 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:24.949 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:24.949 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:24.950 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:24.951 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:24.945+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:25.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:08:25.055 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:25.050+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:25.144 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:25.144 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:25.144 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:25.144 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:25.152 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:25.150+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:25.160 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:25.252 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:25.249+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:25.284 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:25.284 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:25.284 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:25.284 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:25.284 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:25.300 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:25.388 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:25.385+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:25.454 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:25.449+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:25.468 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:25.468 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:25.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:25.468 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:25.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.468 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.635617177999997, 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.473 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.634842829999997, 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.474 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3038 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336620656000001, 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046439165999999997, 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3038 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129535+0000", 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3230000000000001e-06 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129541+0000", 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.054e-06 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129559+0000", 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8329e-05 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129573+0000", 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-14T07:08:25.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175474+0000", 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045901019000000001 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175554+0000", 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0661000000000006e-05 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175973+0000", 2024-09-14T07:08:25.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041842300000000002 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3039 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336562976, 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046967035999999997, 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3039 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129592+0000", 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.206e-06 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129598+0000", 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2340000000000002e-06 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129616+0000", 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8003e-05 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129629+0000", 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2913e-05 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176018+0000", 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046389224999999999 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176108+0000", 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176558+0000", 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044997100000000002 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3040 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336473774, 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047451337000000003, 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3040 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129682+0000", 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.607e-06 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129688+0000", 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129708+0000", 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0047e-05 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129723+0000", 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.502e-05 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176597+0000", 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046873259 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176684+0000", 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7349999999999995e-05 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177132+0000", 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044784099999999998 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3041 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336410471000001, 2024-09-14T07:08:25.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047622767000000003, 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3041 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129745+0000", 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129751+0000", 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129773+0000", 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782999999999999e-05 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129788+0000", 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4606999999999999e-05 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177167+0000", 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047378941000000001 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177253+0000", 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5704999999999993e-05 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177367+0000", 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113861 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3042 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336347032000001, 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048188976000000001, 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3042 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129808+0000", 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:25.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129814+0000", 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7699999999999998e-06 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129832+0000", 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.751e-05 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129844+0000", 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2797e-05 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177436+0000", 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047591574999999997 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177545+0000", 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109446 2024-09-14T07:08:25.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177996+0000", 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045071800000000002 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3043 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336290578, 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048376965000000001, 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.500 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3043 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129865+0000", 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129870+0000", 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129888+0000", 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.755e-05 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129902+0000", 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178049+0000", 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048146648 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178131+0000", 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2619000000000003e-05 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178241+0000", 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109262 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3044 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336218498999999, 2024-09-14T07:08:25.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048575449, 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3044 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129937+0000", 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1930000000000001e-06 2024-09-14T07:08:25.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129943+0000", 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129961+0000", 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8034e-05 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129975+0000", 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3696000000000001e-05 2024-09-14T07:08:25.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178293+0000", 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048318051000000001 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178392+0000", 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9038999999999998e-05 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178511+0000", 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119493 2024-09-14T07:08:25.508 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3045 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336159243000001, 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049079214000000003, 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.509 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3045 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129996+0000", 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1170000000000001e-06 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130002+0000", 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130024+0000", 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1763000000000002e-05 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130059+0000", 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.536e-05 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178555+0000", 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048495599 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178636+0000", 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1112000000000005e-05 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179074+0000", 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043820000000000003 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3046 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.336072567, 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049550174000000002, 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3046 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130083+0000", 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130089+0000", 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130108+0000", 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9046999999999999e-05 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130122+0000", 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4137e-05 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179126+0000", 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049003927000000003 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179221+0000", 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4717999999999997e-05 2024-09-14T07:08:25.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179632+0000", 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041092199999999998 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3047 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.33601187, 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050053944000000003, 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.517 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3047 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:25.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130144+0000", 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3030000000000001e-06 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130149+0000", 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6230000000000002e-06 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130169+0000", 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9386999999999999e-05 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130182+0000", 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3706000000000001e-05 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179677+0000", 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049494109000000001 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179763+0000", 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6473999999999999e-05 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180196+0000", 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043334200000000003 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3048 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.335951951, 2024-09-14T07:08:25.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050582655999999997, 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3048 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130204+0000", 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.266e-06 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130219+0000", 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5549999999999999e-05 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130235+0000", 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6030000000000001e-05 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130249+0000", 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3977e-05 2024-09-14T07:08:25.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180264+0000", 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050014851999999999 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180359+0000", 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5400000000000001e-05 2024-09-14T07:08:25.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180785+0000", 2024-09-14T07:08:25.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042558100000000002 2024-09-14T07:08:25.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.526 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3049 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.335884484999999, 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.051156504999999998, 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3049 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130271+0000", 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130277+0000", 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6370000000000004e-06 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130297+0000", 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.997e-05 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130311+0000", 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4358999999999999e-05 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180833+0000", 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050521863 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180936+0000", 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010296799999999999 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.181426+0000", 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049044799999999999 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.531 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3149 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.320697909, 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046930569999999998, 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.532 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3149 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145458+0000", 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145463+0000", 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6169999999999998e-06 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145486+0000", 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2626e-05 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145500+0000", 2024-09-14T07:08:25.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4243000000000001e-05 2024-09-14T07:08:25.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191669+0000", 2024-09-14T07:08:25.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046169090000000003 2024-09-14T07:08:25.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191811+0000", 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014187100000000001 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192387+0000", 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057567299999999996 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3150 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.320631549, 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047680769999999997, 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3150 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.537 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:25.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145524+0000", 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.39e-06 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145647+0000", 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123242 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145673+0000", 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145711+0000", 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8745999999999998e-05 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192444+0000", 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046732462000000002 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192602+0000", 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015825400000000001 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193203+0000", 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00060150600000000002 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.541 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3172 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.320245678999999, 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048262865000000002, 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3172 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145911+0000", 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145920+0000", 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2969999999999992e-06 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145947+0000", 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7149999999999999e-05 2024-09-14T07:08:25.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145965+0000", 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8295999999999999e-05 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193303+0000", 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047338247 2024-09-14T07:08:25.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193492+0000", 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018843500000000001 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194171+0000", 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067951700000000003 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3173 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.320160323, 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048937952999999999, 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3173 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:25.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145995+0000", 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146021+0000", 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5684e-05 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146044+0000", 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2503000000000001e-05 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146059+0000", 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5043e-05 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194258+0000", 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04819975 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194372+0000", 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113373 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194932+0000", 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056004399999999999 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.623659832, 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:25.552 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:25.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:08:25.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:25.558 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3267316259999999, 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:25.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:08:25.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:08:25.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:25.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:25.564 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:25.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:25.566 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:25.566+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:25.666 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:25.662+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:25.699 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:25.699 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:25.699 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:25.699 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:25.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:25.799 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:25.798+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:25.868 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:25.866+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:25.885 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:25.885 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:25.886 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:25.886 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:25.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:25.991 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:25.990+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:26.043 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:08:26.069 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:26.065+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:26.088 INFO:teuthology.orchestra.run.smithi145.stdout:1007 write 0xc881200 thru 0xc882e74 (0x1c75 bytes) 2024-09-14T07:08:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:26.089 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:26.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:368 3.1f 3:f8928d1a:::rbd_data.1161a9b4f57.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.023530484999998, 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01725641, 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:26.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 368 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:26.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064165+0000", 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.540000000000001e-07 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064170+0000", 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8940000000000001e-06 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064185+0000", 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5182e-05 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064193+0000", 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7999999999999999e-06 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064281+0000", 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7517999999999998e-05 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064365+0000", 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4728000000000006e-05 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064567+0000", 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201757 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068739+0000", 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041723519999999998 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081335+0000", 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012595492999999999 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081366+0000", 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1538999999999997e-05 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081421+0000", 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4193000000000001e-05 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.095 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:373 3.1f 3:fcd26671:::rbd_data.1161a9b4f57.0000000000000045:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.023177746000002, 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017078697, 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 373 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064518+0000", 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064524+0000", 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9839999999999998e-06 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064536+0000", 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2089999999999999e-05 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064544+0000", 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1589999999999993e-06 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065143+0000", 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059867500000000001 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065216+0000", 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3497000000000005e-05 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065416+0000", 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019953899999999999 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068744+0000", 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033285459999999999 2024-09-14T07:08:26.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081472+0000", 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012727815 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081528+0000", 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5958000000000002e-05 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081596+0000", 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7489000000000004e-05 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:386 3.1f 3:fbac3711:::rbd_data.1161a9b4f57.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.021433902999998, 2024-09-14T07:08:26.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015471182999999999, 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 386 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066262+0000", 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5600000000000004e-07 2024-09-14T07:08:26.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066268+0000", 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0809999999999999e-06 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066279+0000", 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1389999999999999e-05 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066286+0000", 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8959999999999997e-06 2024-09-14T07:08:26.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066393+0000", 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107413 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066451+0000", 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7454000000000003e-05 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066556+0000", 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105326 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068748+0000", 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002192285 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081641+0000", 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012892280000000001 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081688+0000", 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081732+0000", 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4132999999999999e-05 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.108 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:389 3.1f 3:fddcb068:::rbd_data.1161a9b4f57.0000000000000060:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.021188086999999, 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015361717, 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 389 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066507+0000", 2024-09-14T07:08:26.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1200000000000002e-07 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066518+0000", 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0913999999999999e-05 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066528+0000", 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066533+0000", 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9910000000000002e-06 2024-09-14T07:08:26.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066729+0000", 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019615400000000001 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066810+0000", 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1149999999999994e-05 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.067039+0000", 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022877600000000001 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068752+0000", 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001713542 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081789+0000", 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013036384999999999 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081828+0000", 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8772000000000001e-05 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081868+0000", 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0762e-05 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:392 3.12 3:4c441206:::rbd_data.1161a9b4f57.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:26.114 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.021050421999998, 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014904545, 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 392 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066644+0000", 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066645+0000", 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066647+0000", 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6079999999999998e-06 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066653+0000", 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.631e-06 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066667+0000", 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4051e-05 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066706+0000", 2024-09-14T07:08:26.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8621999999999997e-05 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066767+0000", 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1723000000000005e-05 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066894+0000", 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012675300000000001 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068486+0000", 2024-09-14T07:08:26.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001591692 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081434+0000", 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012948337000000001 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081490+0000", 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5551999999999999e-05 2024-09-14T07:08:26.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081549+0000", 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9114999999999999e-05 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:286 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619458562999998, 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01542496, 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 286 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468238+0000", 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3960000000000001e-06 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468244+0000", 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468267+0000", 2024-09-14T07:08:26.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3603000000000001e-05 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468278+0000", 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0883000000000001e-05 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483169+0000", 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014890970999999999 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483418+0000", 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024896800000000002 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483661+0000", 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024296499999999999 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:287 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.124 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619401590999999, 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015987674, 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 287 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468294+0000", 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6300000000000004e-07 2024-09-14T07:08:26.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468329+0000", 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4799999999999999e-05 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468349+0000", 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0299999999999999e-05 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468363+0000", 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4284000000000001e-05 2024-09-14T07:08:26.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483821+0000", 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015458120000000001 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484068+0000", 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000246281 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484281+0000", 2024-09-14T07:08:26.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000213026 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:288 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619306891000001, 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016408030000000001, 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 288 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468389+0000", 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.64e-06 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468396+0000", 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468411+0000", 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5153e-05 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468419+0000", 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7039999999999994e-06 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484366+0000", 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015946689999999999 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484617+0000", 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025165400000000003 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484796+0000", 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017846299999999999 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:289 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 122880~6656] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619262912, 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111327999999999, 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 289 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468433+0000", 2024-09-14T07:08:26.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1999999999999999e-07 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468436+0000", 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3969999999999998e-06 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468445+0000", 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9800000000000004e-06 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468455+0000", 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6900000000000004e-06 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484948+0000", 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016493799 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485216+0000", 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026811499999999998 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485543+0000", 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032662700000000002 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:290 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619229865000001, 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017722122999999999, 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 290 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:26.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4000000000000001e-07 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468469+0000", 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7529999999999999e-06 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468478+0000", 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6799999999999999e-06 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468485+0000", 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.153e-06 2024-09-14T07:08:26.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485735+0000", 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017249970999999999 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485956+0000", 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022074799999999999 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486187+0000", 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023117799999999999 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:291 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:26.143 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619198788999999, 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018193804000000001, 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 291 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468497+0000", 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468502+0000", 2024-09-14T07:08:26.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5439999999999998e-06 2024-09-14T07:08:26.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468531+0000", 2024-09-14T07:08:26.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8898999999999999e-05 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468543+0000", 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1756999999999999e-05 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486295+0000", 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017751423999999998 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486492+0000", 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019711899999999999 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486690+0000", 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019808099999999999 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:292 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619091986000001, 2024-09-14T07:08:26.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018553717000000001, 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 292 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468604+0000", 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5999999999999999e-06 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468611+0000", 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7800000000000003e-06 2024-09-14T07:08:26.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468633+0000", 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1710000000000001e-05 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468647+0000", 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4209999999999999e-05 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486764+0000", 2024-09-14T07:08:26.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018116538000000001 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486948+0000", 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018412999999999999 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487156+0000", 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020874899999999999 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:293 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.619025424, 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019096847, 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 293 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.153 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468671+0000", 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468676+0000", 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.857e-06 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468694+0000", 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7629999999999999e-05 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468712+0000", 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7762999999999999e-05 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487288+0000", 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018576294 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487555+0000", 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026692399999999998 2024-09-14T07:08:26.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487766+0000", 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021110899999999999 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7989537440000003, 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:08:26.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.158 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:08:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7694461130000008, 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:08:26.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:08:26.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7607987539999996, 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:08:26.168 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:26.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:08:26.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:341 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:b [sparse-read 9764864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.4723886960000003, 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015588305, 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 341 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615307+0000", 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615312+0000", 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615331+0000", 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8542e-05 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615346+0000", 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4826e-05 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.629124+0000", 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013778367999999999 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.629369+0000", 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000244842 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.630894+0000", 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015252600000000001 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:26.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.094706414, 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.9279771879999998, 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:08:26.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:08:26.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7460456739999999, 2024-09-14T07:08:26.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:08:26.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:08:26.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:08:26.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:08:26.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.191 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:26.192 INFO:teuthology.orchestra.run.smithi145.stdout:1008 read 0xbd3f200 thru 0xbd43093 (0x3e94 bytes) 2024-09-14T07:08:26.192 INFO:teuthology.orchestra.run.smithi145.stdout:1009 punch from 0x2319600 to 0x2326800, (0xd200 bytes) 2024-09-14T07:08:26.193 INFO:teuthology.orchestra.run.smithi145.stdout:1010 read 0x9499400 thru 0x949e093 (0x4c94 bytes) 2024-09-14T07:08:26.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:26.194 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:26.189+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:26.241 INFO:teuthology.orchestra.run.smithi145.stdout:1013 write 0x679c000 thru 0x67a1a1f (0x5a20 bytes) 2024-09-14T07:08:26.242 INFO:teuthology.orchestra.run.smithi145.stdout:1016 read 0x65b200 thru 0x666501 (0xb302 bytes) 2024-09-14T07:08:26.244 INFO:teuthology.orchestra.run.smithi145.stdout:1017 write 0x6810200 thru 0x681a758 (0xa559 bytes) 2024-09-14T07:08:26.245 INFO:teuthology.orchestra.run.smithi145.stdout:1018 read 0x758a00 thru 0x75b8c2 (0x2ec3 bytes) 2024-09-14T07:08:26.246 INFO:teuthology.orchestra.run.smithi145.stdout:1019 read 0x1c3f200 thru 0x1c41a56 (0x2857 bytes) 2024-09-14T07:08:26.248 INFO:teuthology.orchestra.run.smithi145.stdout:1020 read 0x3aeea00 thru 0x3af8708 (0x9d09 bytes) 2024-09-14T07:08:26.250 INFO:teuthology.orchestra.run.smithi145.stdout:1022 write 0x9e43200 thru 0x9e4b5af (0x83b0 bytes) 2024-09-14T07:08:26.251 INFO:teuthology.orchestra.run.smithi145.stdout:1024 read 0x85a9600 thru 0x85af35d (0x5d5e bytes) 2024-09-14T07:08:26.252 INFO:teuthology.orchestra.run.smithi145.stdout:1025 write 0x90be400 thru 0x90c5ad6 (0x76d7 bytes) 2024-09-14T07:08:26.254 INFO:teuthology.orchestra.run.smithi145.stdout:1028 punch from 0x93e0000 to 0x93e5600, (0x5600 bytes) 2024-09-14T07:08:26.256 INFO:teuthology.orchestra.run.smithi145.stdout:1029 read 0x53e3800 thru 0x53ed8af (0xa0b0 bytes) 2024-09-14T07:08:26.262 INFO:teuthology.orchestra.run.smithi145.stdout:1030 write 0x7a3200 thru 0x7a8896 (0x5697 bytes) 2024-09-14T07:08:26.265 INFO:teuthology.orchestra.run.smithi145.stdout:1033 read 0x2b50000 thru 0x2b55090 (0x5091 bytes) 2024-09-14T07:08:26.284 INFO:teuthology.orchestra.run.smithi145.stdout:1034 trunc from 0xc883000 to 0x2eeba00 2024-09-14T07:08:26.293 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:26.289+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:26.340 INFO:teuthology.orchestra.run.smithi145.stdout:1035 read 0x1839200 thru 0x18458a0 (0xc6a1 bytes) 2024-09-14T07:08:26.349 INFO:teuthology.orchestra.run.smithi145.stdout:1036 trunc from 0x2eeba00 to 0x4d8e200 2024-09-14T07:08:26.369 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:26.370 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:26.384 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:26.411 INFO:teuthology.orchestra.run.smithi145.stdout:1037 write 0xad39e00 thru 0xad3da28 (0x3c29 bytes) 2024-09-14T07:08:26.413 INFO:teuthology.orchestra.run.smithi145.stdout:1041 read 0x44a9800 thru 0x44b0ed7 (0x76d8 bytes) 2024-09-14T07:08:26.435 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:26.434+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:26.441 INFO:teuthology.orchestra.run.smithi145.stdout:1042 write 0xaeff600 thru 0xaf02e9b (0x389c bytes) 2024-09-14T07:08:26.444 INFO:teuthology.orchestra.run.smithi145.stdout:1044 punch from 0x1ee7a00 to 0x1ee8400, (0xa00 bytes) 2024-09-14T07:08:26.453 INFO:teuthology.orchestra.run.smithi145.stdout:1045 trunc from 0xaf03000 to 0x6aea000 2024-09-14T07:08:26.494 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:26.489+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:26.517 INFO:teuthology.orchestra.run.smithi145.stdout:1047 write 0xa72e400 thru 0xa73abe0 (0xc7e1 bytes) 2024-09-14T07:08:26.519 INFO:teuthology.orchestra.run.smithi145.stdout:1048 write 0x95d7a00 thru 0x95dfb49 (0x814a bytes) 2024-09-14T07:08:26.520 INFO:teuthology.orchestra.run.smithi145.stdout:1049 read 0x992f200 thru 0x993e100 (0xef01 bytes) 2024-09-14T07:08:26.521 INFO:teuthology.orchestra.run.smithi145.stdout:1050 write 0x63f2800 thru 0x63f92a4 (0x6aa5 bytes) 2024-09-14T07:08:26.525 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:26.525 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:26.526 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:26.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:26.555 INFO:teuthology.orchestra.run.smithi145.stdout:1054 write 0xb4bb000 thru 0xb4c5d28 (0xad29 bytes) 2024-09-14T07:08:26.557 INFO:teuthology.orchestra.run.smithi145.stdout:1055 punch from 0x1f68400 to 0x1f69400, (0x1000 bytes) 2024-09-14T07:08:26.558 INFO:teuthology.orchestra.run.smithi145.stdout:1058 read 0xae7a200 thru 0xae8330f (0x9110 bytes) 2024-09-14T07:08:26.558 INFO:teuthology.orchestra.run.smithi145.stdout:1059 write 0x2370600 thru 0x2380295 (0xfc96 bytes) 2024-09-14T07:08:26.577 INFO:teuthology.orchestra.run.smithi145.stdout:1068 trunc from 0xb4c5e00 to 0x226ae00 2024-09-14T07:08:26.631 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:26.630+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:26.653 INFO:teuthology.orchestra.run.smithi145.stdout:1070 write 0x6583c00 thru 0x658c8af (0x8cb0 bytes) 2024-09-14T07:08:26.656 INFO:teuthology.orchestra.run.smithi145.stdout:1074 read 0x757200 thru 0x76289c (0xb69d bytes) 2024-09-14T07:08:26.657 INFO:teuthology.orchestra.run.smithi145.stdout:1075 read 0x1e2fe00 thru 0x1e35eaa (0x60ab bytes) 2024-09-14T07:08:26.658 INFO:teuthology.orchestra.run.smithi145.stdout:1078 read 0x29b6600 thru 0x29c328a (0xcc8b bytes) 2024-09-14T07:08:26.659 INFO:teuthology.orchestra.run.smithi145.stdout:1081 punch from 0x6f9600 to 0x700800, (0x7200 bytes) 2024-09-14T07:08:26.692 INFO:teuthology.orchestra.run.smithi145.stdout:1083 write 0xd973200 thru 0xd97cb8b (0x998c bytes) 2024-09-14T07:08:26.694 INFO:teuthology.orchestra.run.smithi145.stdout:1084 write 0x6d38800 thru 0x6d40f92 (0x8793 bytes) 2024-09-14T07:08:26.695 INFO:teuthology.orchestra.run.smithi145.stdout:1085 punch from 0x6943e00 to 0x694b800, (0x7a00 bytes) 2024-09-14T07:08:26.695 INFO:teuthology.orchestra.run.smithi145.stdout:1088 read 0x5051400 thru 0x5054102 (0x2d03 bytes) 2024-09-14T07:08:26.696 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:26.693+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:26.697 INFO:teuthology.orchestra.run.smithi145.stdout:1089 punch from 0x6e4e600 to 0x6e5c800, (0xe200 bytes) 2024-09-14T07:08:26.697 INFO:teuthology.orchestra.run.smithi145.stdout:1094 write 0xa26a000 thru 0xa274cf0 (0xacf1 bytes) 2024-09-14T07:08:26.698 INFO:teuthology.orchestra.run.smithi145.stdout:1095 write 0x76aa000 thru 0x76b87e2 (0xe7e3 bytes) 2024-09-14T07:08:26.701 INFO:teuthology.orchestra.run.smithi145.stdout:1098 trunc from 0xd97cc00 to 0xc871000 2024-09-14T07:08:26.727 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:295 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.255842812000001, 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028784614, 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.728 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 295 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469603+0000", 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469619+0000", 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6163000000000001e-05 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469646+0000", 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6040000000000001e-05 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469660+0000", 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4843e-05 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488319+0000", 2024-09-14T07:08:26.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018658726 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488567+0000", 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024742500000000001 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.489393+0000", 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082641099999999999 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493423+0000", 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040299050000000003 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498065+0000", 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046422310000000001 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498254+0000", 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000189278 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498385+0000", 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013043899999999999 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1514 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5096481349999991, 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028657183999999999, 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1514 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2300000000000002e-07 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215804+0000", 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.602e-06 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215811+0000", 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8190000000000003e-06 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215816+0000", 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2229999999999998e-06 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244228+0000", 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028411750999999999 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244350+0000", 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012267199999999999 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244452+0000", 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010169400000000001 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1515 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5096200310000008, 2024-09-14T07:08:26.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843423999999999, 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1515 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215824+0000", 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215826+0000", 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-06 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215832+0000", 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.947e-06 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215837+0000", 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6570000000000002e-06 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244479+0000", 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028642223000000001 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244559+0000", 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9443000000000005e-05 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244666+0000", 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107538 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1517 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10027008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.742 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5095994990000001, 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029071107999999998, 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1517 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215844+0000", 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215846+0000", 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.821e-06 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215852+0000", 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215857+0000", 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6099999999999997e-06 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244719+0000", 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028862291000000002 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244803+0000", 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3523999999999995e-05 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244915+0000", 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111612 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.746 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1518 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5095788209999998, 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029262857, 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1518 2024-09-14T07:08:26.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215865+0000", 2024-09-14T07:08:26.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5699999999999998e-07 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215867+0000", 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215878+0000", 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1399e-05 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215883+0000", 2024-09-14T07:08:26.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244948+0000", 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029064834000000001 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245029+0000", 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0693999999999997e-05 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245127+0000", 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8218000000000001e-05 2024-09-14T07:08:26.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1520 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5095518660000007, 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029524788999999999, 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1520 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215892+0000", 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215894+0000", 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6860000000000002e-06 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215902+0000", 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7929999999999998e-06 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215907+0000", 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245181+0000", 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029273648999999999 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245287+0000", 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010595299999999999 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245416+0000", 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012906300000000001 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1522 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5095263550000002, 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029733393, 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1522 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:26.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9599999999999999e-07 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215919+0000", 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2749999999999998e-06 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215931+0000", 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215937+0000", 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245451+0000", 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029513475000000001 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245537+0000", 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5975000000000003e-05 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245650+0000", 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011319099999999999 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1523 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13959168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.759 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5094929459999999, 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029960937999999999, 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1523 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215954+0000", 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3689999999999998e-06 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215961+0000", 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2420000000000003e-06 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215967+0000", 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8329999999999999e-06 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245715+0000", 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029748177000000001 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245798+0000", 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3338999999999996e-05 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245911+0000", 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112507 2024-09-14T07:08:26.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1524 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5094675510000002, 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030280225000000001, 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1524 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215976+0000", 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215978+0000", 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0509999999999999e-06 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215985+0000", 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8120000000000002e-06 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215990+0000", 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5729999999999998e-06 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246034+0000", 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030043767999999998 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246121+0000", 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7093000000000001e-05 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246256+0000", 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013446899999999999 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1525 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5094109769999999, 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030480007, 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1525 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:26.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216033+0000", 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0800000000000004e-07 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216036+0000", 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8600000000000003e-06 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216045+0000", 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3720000000000005e-06 2024-09-14T07:08:26.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216052+0000", 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246311+0000", 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030259496 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246397+0000", 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5339000000000004e-05 2024-09-14T07:08:26.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246512+0000", 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000115393 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1664 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.504093224, 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028510918999999999, 2024-09-14T07:08:26.772 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1664 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221354+0000", 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2339999999999999e-06 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221364+0000", 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0445000000000001e-05 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221373+0000", 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4950000000000008e-06 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249615+0000", 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028241973 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249750+0000", 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013522499999999999 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249861+0000", 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110908 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1665 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:26.776 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5040591580000005, 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028695215, 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1665 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9699999999999996e-07 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221387+0000", 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221395+0000", 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0399999999999993e-06 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221401+0000", 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.806e-06 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249893+0000", 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028491951000000001 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249971+0000", 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8419999999999998e-05 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250079+0000", 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010798600000000001 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1667 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5040333029999999, 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028871159, 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1667 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221413+0000", 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.396e-06 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221422+0000", 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2250000000000006e-06 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221428+0000", 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0839999999999997e-06 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250112+0000", 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028684345 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250191+0000", 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8571000000000004e-05 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250281+0000", 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0094999999999997e-05 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1668 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5039938849999999, 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02902478, 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1668 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221450+0000", 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1399999999999997e-07 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221453+0000", 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221462+0000", 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.884e-06 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221468+0000", 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5390000000000004e-06 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250311+0000", 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843160999999999 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250389+0000", 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7550000000000001e-05 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250474+0000", 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4982999999999996e-05 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1669 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:26.789 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5039405909999992, 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029194801999999999, 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1669 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:26.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221503+0000", 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221506+0000", 2024-09-14T07:08:26.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221516+0000", 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9450000000000005e-06 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221523+0000", 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250506+0000", 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028983992 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250609+0000", 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010213399999999999 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250697+0000", 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8551e-05 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1670 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5039103209999993, 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029401132999999999, 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1670 2024-09-14T07:08:26.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:26.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221536+0000", 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5550000000000001e-06 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221543+0000", 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4939999999999997e-06 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250744+0000", 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029195045999999999 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250836+0000", 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1589999999999996e-05 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250934+0000", 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7826999999999997e-05 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1904 3.b 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9175040~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4365503440000005, 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.101088252, 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1904 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288894+0000", 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.666e-06 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315149+0000", 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026254493 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315177+0000", 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-09-14T07:08:26.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315200+0000", 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2949e-05 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315307+0000", 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107222 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315483+0000", 2024-09-14T07:08:26.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017589 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.317437+0000", 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019539890000000002 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.384167+0000", 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066730085999999994 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:26.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389856+0000", 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056887079999999998 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389925+0000", 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8977e-05 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389981+0000", 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6158e-05 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1907 3.1a e43/21 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head v 43'107, mlcod=42'105)", 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4032650560000004, 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.068409926999999995, 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8299999999999998e-07 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331614+0000", 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094355310000000005 2024-09-14T07:08:26.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331636+0000", 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1804e-05 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331668+0000", 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2094999999999997e-05 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331759+0000", 2024-09-14T07:08:26.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0901000000000003e-05 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331787+0000", 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8750000000000001e-05 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390536+0000", 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058748732999999997 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390588+0000", 2024-09-14T07:08:26.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1829999999999997e-05 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1911 3.8 e43/25 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head v 43'64, mlcod=42'61)", 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3760438270000002, 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.060945845999999998, 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349401+0000", 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.846e-06 2024-09-14T07:08:26.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354404+0000", 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005003146 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354427+0000", 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2617999999999999e-05 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354440+0000", 2024-09-14T07:08:26.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3559e-05 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354525+0000", 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4986000000000003e-05 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354551+0000", 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5276999999999999e-05 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410278+0000", 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055727499999999999 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410345+0000", 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6914000000000003e-05 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:26.812 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.0456579760000002, 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:26.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:08:26.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:26.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:26.817 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:26.817 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:26.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:26.818 INFO:teuthology.orchestra.run.smithi145.stdout:1106 trunc from 0xc871000 to 0x9f78c00 2024-09-14T07:08:26.818 INFO:teuthology.orchestra.run.smithi145.stdout:1108 clone 10 order 25 su 1048576 sc 3 2024-09-14T07:08:26.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:26.819 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:26.818+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:26.919 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:26.918+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:26.990 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:26.991 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:26.991 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:26.991 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:27.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:27.021 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:27.022 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:27.022 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:27.018+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:27.038 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:27.060 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:27.081 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:27.104 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:27.105 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:27.121 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:27.117+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:27.123 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:27.123 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:27.142 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:27.142 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:27.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:27.247 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:27.246+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:27.322 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:27.322 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:27.322 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:27.317+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:27.337 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:27.424 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:27.422+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:27.469 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:27.469 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:27.483 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:27.531 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:27.529+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:27.605 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:27.605 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:27.619 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:27.664 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:27.661+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:27.727 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:27.725+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:27.781 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:27.781 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:27.796 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:27.839 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:27.837+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:27.929 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:27.925+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:27.956 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:27.956 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:27.970 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:28.030 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:28.025+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:28.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:08:28.097 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:28.097 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:28.111 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:28.131 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:28.129+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:28.231 INFO:teuthology.orchestra.run.smithi145.stdout:flattening image image_client.0-clone9 2024-09-14T07:08:28.232 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:28.229+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:28.275 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:28.276 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:28.289 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:28.332 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:28.329+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:28.433 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:28.429+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:28.452 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:28.452 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:28.466 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:28.534 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:28.529+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:28.580 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:28.580 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:28.593 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:28.636 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:28.633+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:28.737 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:28.733+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:28.753 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:28.753 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:28.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:28.839 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:28.833+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:28.906 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:28.906 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:28.912 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T07:08:28.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:28.971 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:28.969+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:29.040 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:29.037+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:29.055 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 2] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:29.055 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:29.055 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-14T07:08:29.055 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-14T07:08:29.062 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:29.062 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:29.073 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:08:29.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:29.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:29.143 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:29.141+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:29.243 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:29.241+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:29.279 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:29.279 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:29.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:29.319 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:29.319 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:29.341 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:29.344 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:29.341+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:29.376 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:29.377 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:29.415 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:29.431 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:29.432 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:29.432 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:29.432 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:29.444 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:29.441+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:29.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:29.451 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:29.475 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:29.475 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:29.481 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:08:29.477+0000 7f1e6b537640 -1 mon.a@0(leader).osd e53 definitely_dead 0 2024-09-14T07:08:29.545 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:29.541+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:29.585 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:29.585 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:29.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:29.646 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:29.641+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:29.724 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:295 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.254411595000001, 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028784614, 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 295 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:29.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469603+0000", 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469619+0000", 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6163000000000001e-05 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469646+0000", 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6040000000000001e-05 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469660+0000", 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4843e-05 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488319+0000", 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018658726 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488567+0000", 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024742500000000001 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.489393+0000", 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082641099999999999 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493423+0000", 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040299050000000003 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498065+0000", 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046422310000000001 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498254+0000", 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000189278 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498385+0000", 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013043899999999999 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1514 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508216918, 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028657183999999999, 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1514 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.731 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:29.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2300000000000002e-07 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215804+0000", 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.602e-06 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215811+0000", 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8190000000000003e-06 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215816+0000", 2024-09-14T07:08:29.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2229999999999998e-06 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244228+0000", 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028411750999999999 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244350+0000", 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012267199999999999 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244452+0000", 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010169400000000001 2024-09-14T07:08:29.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1515 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508188814, 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843423999999999, 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1515 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215824+0000", 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215826+0000", 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-06 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215832+0000", 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.947e-06 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215837+0000", 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6570000000000002e-06 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244479+0000", 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028642223000000001 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244559+0000", 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9443000000000005e-05 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244666+0000", 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107538 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1517 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10027008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508168282, 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029071107999999998, 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1517 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:29.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215844+0000", 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215846+0000", 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.821e-06 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215852+0000", 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215857+0000", 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6099999999999997e-06 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244719+0000", 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028862291000000002 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244803+0000", 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3523999999999995e-05 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244915+0000", 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111612 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1518 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:29.743 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508147603999999, 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029262857, 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1518 2024-09-14T07:08:29.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215865+0000", 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5699999999999998e-07 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215867+0000", 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215878+0000", 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1399e-05 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215883+0000", 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244948+0000", 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029064834000000001 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245029+0000", 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0693999999999997e-05 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245127+0000", 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8218000000000001e-05 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1520 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508120649, 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029524788999999999, 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1520 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215892+0000", 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215894+0000", 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6860000000000002e-06 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215902+0000", 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7929999999999998e-06 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215907+0000", 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245181+0000", 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029273648999999999 2024-09-14T07:08:29.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245287+0000", 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010595299999999999 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245416+0000", 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012906300000000001 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.752 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1522 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508095138, 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029733393, 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1522 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9599999999999999e-07 2024-09-14T07:08:29.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215919+0000", 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2749999999999998e-06 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215931+0000", 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215937+0000", 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245451+0000", 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029513475000000001 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245537+0000", 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5975000000000003e-05 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245650+0000", 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011319099999999999 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.756 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1523 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13959168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508061729, 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029960937999999999, 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1523 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215954+0000", 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3689999999999998e-06 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215961+0000", 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2420000000000003e-06 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215967+0000", 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8329999999999999e-06 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245715+0000", 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029748177000000001 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245798+0000", 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3338999999999996e-05 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245911+0000", 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112507 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1524 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.508036334, 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030280225000000001, 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1524 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:29.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215976+0000", 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215978+0000", 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0509999999999999e-06 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215985+0000", 2024-09-14T07:08:29.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8120000000000002e-06 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215990+0000", 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5729999999999998e-06 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246034+0000", 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030043767999999998 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246121+0000", 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7093000000000001e-05 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246256+0000", 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013446899999999999 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1525 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.765 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.50797976, 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030480007, 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1525 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216033+0000", 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0800000000000004e-07 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216036+0000", 2024-09-14T07:08:29.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8600000000000003e-06 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216045+0000", 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3720000000000005e-06 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216052+0000", 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246311+0000", 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030259496 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246397+0000", 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5339000000000004e-05 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246512+0000", 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000115393 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.769 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1664 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.502662007, 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028510918999999999, 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1664 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221354+0000", 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2339999999999999e-06 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221364+0000", 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0445000000000001e-05 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221373+0000", 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4950000000000008e-06 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249615+0000", 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028241973 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249750+0000", 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013522499999999999 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249861+0000", 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110908 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1665 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.502627941, 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028695215, 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1665 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9699999999999996e-07 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221387+0000", 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221395+0000", 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0399999999999993e-06 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221401+0000", 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.806e-06 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249893+0000", 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028491951000000001 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249971+0000", 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8419999999999998e-05 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250079+0000", 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010798600000000001 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1667 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.502602086, 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028871159, 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1667 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221413+0000", 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.396e-06 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221422+0000", 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2250000000000006e-06 2024-09-14T07:08:29.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221428+0000", 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0839999999999997e-06 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250112+0000", 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028684345 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250191+0000", 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8571000000000004e-05 2024-09-14T07:08:29.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250281+0000", 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0094999999999997e-05 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1668 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.502562667999999, 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02902478, 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1668 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221450+0000", 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1399999999999997e-07 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221453+0000", 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221462+0000", 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.884e-06 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221468+0000", 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5390000000000004e-06 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250311+0000", 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843160999999999 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250389+0000", 2024-09-14T07:08:29.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7550000000000001e-05 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250474+0000", 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4982999999999996e-05 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1669 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.502509374000001, 2024-09-14T07:08:29.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029194801999999999, 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1669 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221503+0000", 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-09-14T07:08:29.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221506+0000", 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221516+0000", 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9450000000000005e-06 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221523+0000", 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250506+0000", 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028983992 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250609+0000", 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010213399999999999 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250697+0000", 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8551e-05 2024-09-14T07:08:29.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1670 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.502479104000001, 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029401132999999999, 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1670 2024-09-14T07:08:29.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:29.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221536+0000", 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5550000000000001e-06 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221543+0000", 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4939999999999997e-06 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:08:29.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250744+0000", 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029195045999999999 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250836+0000", 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1589999999999996e-05 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250934+0000", 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7826999999999997e-05 2024-09-14T07:08:29.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1904 3.b 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9175040~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.435119127, 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.101088252, 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1904 2024-09-14T07:08:29.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288894+0000", 2024-09-14T07:08:29.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.666e-06 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315149+0000", 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026254493 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315177+0000", 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315200+0000", 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2949e-05 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315307+0000", 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107222 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315483+0000", 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017589 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.317437+0000", 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019539890000000002 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.384167+0000", 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066730085999999994 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389856+0000", 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056887079999999998 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389925+0000", 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8977e-05 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389981+0000", 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6158e-05 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1907 3.1a e43/21 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head v 43'107, mlcod=42'105)", 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:29.800 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.401833839, 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.068409926999999995, 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8299999999999998e-07 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331614+0000", 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094355310000000005 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331636+0000", 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1804e-05 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331668+0000", 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2094999999999997e-05 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331759+0000", 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0901000000000003e-05 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331787+0000", 2024-09-14T07:08:29.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8750000000000001e-05 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390536+0000", 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058748732999999997 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390588+0000", 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1829999999999997e-05 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1911 3.8 e43/25 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head v 43'64, mlcod=42'61)", 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.37461261, 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.060945845999999998, 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349401+0000", 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.846e-06 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354404+0000", 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005003146 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354427+0000", 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2617999999999999e-05 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354440+0000", 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3559e-05 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354525+0000", 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4986000000000003e-05 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354551+0000", 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5276999999999999e-05 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410278+0000", 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055727499999999999 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410345+0000", 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6914000000000003e-05 2024-09-14T07:08:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.0442267590000007, 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:29.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:29.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:29.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:29.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:29.814 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:29.815 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:29.809+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:29.898 INFO:teuthology.orchestra.run.smithi145.stdout:1110 write 0x1466000 thru 0x146d231 (0x7232 bytes) 2024-09-14T07:08:29.915 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:29.913+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:29.921 INFO:teuthology.orchestra.run.smithi145.stdout:1111 trunc from 0x9f78c00 to 0xde60400 2024-09-14T07:08:29.992 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:29.992 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:29.993 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:29.994 INFO:teuthology.orchestra.run.smithi145.stdout:1116 read 0x599ac00 thru 0x59a1392 (0x6793 bytes) 2024-09-14T07:08:29.999 INFO:teuthology.orchestra.run.smithi145.stdout:1121 read 0x2cb800 thru 0x2ceb43 (0x3344 bytes) 2024-09-14T07:08:30.000 INFO:teuthology.orchestra.run.smithi145.stdout:1124 write 0x9a17000 thru 0x9a24432 (0xd433 bytes) 2024-09-14T07:08:30.002 INFO:teuthology.orchestra.run.smithi145.stdout:1125 write 0x920fa00 thru 0x921a66a (0xac6b bytes) 2024-09-14T07:08:30.004 INFO:teuthology.orchestra.run.smithi145.stdout:1130 write 0xb46b800 thru 0xb4719fa (0x61fb bytes) 2024-09-14T07:08:30.004 INFO:teuthology.orchestra.run.smithi145.stdout:1131 punch from 0x8772a00 to 0x8774a00, (0x2000 bytes) 2024-09-14T07:08:30.006 INFO:teuthology.orchestra.run.smithi145.stdout:1132 trunc from 0xde60400 to 0xd642c00 2024-09-14T07:08:30.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:30.055 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:30.053+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:30.055 INFO:teuthology.orchestra.run.smithi145.stdout:1137 read 0x3480e00 thru 0x3481ed7 (0x10d8 bytes) 2024-09-14T07:08:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.4. 2024-09-14T07:08:30.103 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-14T07:08:30.130 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:30.130 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:30.130 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:30.131 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:30.125+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:30.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:30.154 INFO:teuthology.orchestra.run.smithi145.stdout:1141 trunc from 0xd642c00 to 0x1aa1400 2024-09-14T07:08:30.218 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:30.213+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:30.309 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:30.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1538 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086304905, 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054129517000000002, 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1538 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.310 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220949+0000", 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-06 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220959+0000", 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0932e-05 2024-09-14T07:08:30.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220965+0000", 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.49e-06 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274976+0000", 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054011221999999998 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275019+0000", 2024-09-14T07:08:30.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2889000000000002e-05 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275076+0000", 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6455000000000001e-05 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1539 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086278189, 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054229843, 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1539 2024-09-14T07:08:30.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220976+0000", 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-06 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:30.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1069999999999999e-06 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220988+0000", 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0119999999999996e-06 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275106+0000", 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054118053999999999 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275149+0000", 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2830000000000002e-05 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275203+0000", 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4044e-05 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1540 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086255873000001, 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05433267, 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1540 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220996+0000", 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220998+0000", 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9759999999999998e-06 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221009+0000", 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.242e-06 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275233+0000", 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054223461000000001 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275275+0000", 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2726999999999999e-05 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275328+0000", 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2332e-05 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1541 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086233676999999, 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054437230000000003, 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1541 2024-09-14T07:08:30.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:30.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221018+0000", 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2e-07 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221020+0000", 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3530000000000002e-06 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221026+0000", 2024-09-14T07:08:30.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3110000000000002e-06 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221031+0000", 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275358+0000", 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054326722000000001 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275401+0000", 2024-09-14T07:08:30.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2849000000000003e-05 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275454+0000", 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3844000000000002e-05 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1542 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086213262999999, 2024-09-14T07:08:30.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054541385999999997, 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1542 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:30.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3700000000000001e-07 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221040+0000", 2024-09-14T07:08:30.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8059999999999999e-06 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221046+0000", 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1009999999999997e-06 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221051+0000", 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6940000000000003e-06 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275484+0000", 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054433003000000001 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275527+0000", 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3031999999999997e-05 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275579+0000", 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2413000000000001e-05 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1543 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086193507000001, 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054648398000000001, 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1543 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221058+0000", 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4299999999999999e-07 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221060+0000", 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7990000000000001e-06 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221066+0000", 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221070+0000", 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7010000000000004e-06 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275609+0000", 2024-09-14T07:08:30.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054538968 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275652+0000", 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2864000000000002e-05 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275706+0000", 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3597000000000002e-05 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1545 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086173841000001, 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054855590000000003, 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1545 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4499999999999998e-07 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221085+0000", 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0330000000000008e-06 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221092+0000", 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.793e-06 2024-09-14T07:08:30.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221098+0000", 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3090000000000003e-06 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275744+0000", 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054646404000000003 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275821+0000", 2024-09-14T07:08:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6793999999999997e-05 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275933+0000", 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111912 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.342 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1583 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 524288~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086145677999999, 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055023028000000002, 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1583 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.343 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221106+0000", 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221108+0000", 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0229999999999999e-06 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221121+0000", 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3645000000000001e-05 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221127+0000", 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.541e-06 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275735+0000", 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054608372000000002 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275817+0000", 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2083999999999995e-05 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276128+0000", 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031090000000000002 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.347 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1585 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086115844, 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055153874999999998, 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1585 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221136+0000", 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7300000000000001e-07 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221138+0000", 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221144+0000", 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5100000000000004e-06 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221149+0000", 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.843e-06 2024-09-14T07:08:30.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276165+0000", 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055015677999999998 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276231+0000", 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5959999999999999e-05 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276289+0000", 2024-09-14T07:08:30.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8100000000000003e-05 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1586 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.08609223, 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055262004000000003, 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1586 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221166+0000", 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3180000000000001e-06 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221173+0000", 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:08:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221178+0000", 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2050000000000001e-06 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276321+0000", 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055142663000000001 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276364+0000", 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3615000000000002e-05 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276421+0000", 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6453000000000003e-05 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1588 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086064993000001, 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055364653999999999, 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1588 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221189+0000", 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3269999999999999e-06 2024-09-14T07:08:30.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221195+0000", 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2169999999999999e-06 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221200+0000", 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276451+0000", 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055251452999999999 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276497+0000", 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5114000000000002e-05 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276551+0000", 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4079999999999997e-05 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1589 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086044329, 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055473737000000002, 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1589 2024-09-14T07:08:30.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:30.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221209+0000", 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9269999999999999e-06 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221215+0000", 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3069999999999999e-06 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221220+0000", 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6369999999999998e-06 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276582+0000", 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055361856000000001 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276625+0000", 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2803000000000003e-05 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276680+0000", 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5789999999999999e-05 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1591 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.086023968999999, 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055584770999999998, 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1591 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:30.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222763+0000", 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001535914 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222789+0000", 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6194e-05 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222862+0000", 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2529999999999998e-05 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222953+0000", 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1185000000000002e-05 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276713+0000", 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053760371000000001 2024-09-14T07:08:30.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276758+0000", 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4932000000000003e-05 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276812+0000", 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3644999999999999e-05 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.369 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1593 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 6225920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.084253294, 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054086803000000003, 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1593 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223000+0000", 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.289e-06 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223007+0000", 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.058e-06 2024-09-14T07:08:30.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223013+0000", 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5779999999999998e-06 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276973+0000", 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053960263000000001 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277028+0000", 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000002e-05 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277084+0000", 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6357999999999998e-05 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1594 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.377 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.084229429000001, 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054193419, 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1594 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223035+0000", 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2994e-05 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223043+0000", 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6539999999999999e-06 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223048+0000", 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6149999999999996e-06 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277114+0000", 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054066256 2024-09-14T07:08:30.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277157+0000", 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2629000000000001e-05 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277215+0000", 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7803999999999997e-05 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.381 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1598 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.083609874, 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054092236000000002, 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1598 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224174+0000", 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000533265 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224179+0000", 2024-09-14T07:08:30.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2420000000000002e-06 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224190+0000", 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-05 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224197+0000", 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3549999999999999e-06 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277635+0000", 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053437314999999999 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277679+0000", 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4628e-05 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277733+0000", 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4131000000000001e-05 2024-09-14T07:08:30.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.006944238000001, 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.391 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.961953932, 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:08:30.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:08:30.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:08:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:08:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6600214760000007, 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:08:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.3386937420000002, 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:08:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:08:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:08:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:08:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:08:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:08:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:30.409 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:30.411 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:30.411 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:30.405+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:30.511 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:30.509+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:30.542 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:30.542 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:30.557 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:30.612 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:30.609+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:30.680 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:30.680 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:30.680 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:30.680 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:30.680 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:30.695 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:30.713 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:30.709+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:30.815 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:30.813+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:30.857 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:30.857 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 58.556022327000001, 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:30.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:30.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.864 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1502 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.635657967, 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042080580999999999, 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1502 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220869+0000", 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2300000000000001e-06 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220888+0000", 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8607e-05 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220907+0000", 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9422000000000001e-05 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220922+0000", 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4375999999999999e-05 2024-09-14T07:08:30.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262777+0000", 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855537999999998 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262833+0000", 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5729000000000001e-05 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262949+0000", 2024-09-14T07:08:30.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115679 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1503 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.635583154999999, 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042296750000000001, 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1503 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220945+0000", 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.013e-06 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220951+0000", 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.806e-06 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220970+0000", 2024-09-14T07:08:30.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8649999999999999e-05 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3964999999999999e-05 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262989+0000", 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042005845 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263116+0000", 2024-09-14T07:08:30.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012688 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263240+0000", 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012359099999999999 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1504 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.635524030999999, 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042493942999999999, 2024-09-14T07:08:30.873 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1504 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:30.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.063e-06 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221011+0000", 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8059999999999997e-06 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221030+0000", 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8652999999999999e-05 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221044+0000", 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3879999999999999e-05 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263275+0000", 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042231869999999998 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263365+0000", 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9813000000000002e-05 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263496+0000", 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000130858 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1634 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.632329969000001, 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041961863000000002, 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1634 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224198+0000", 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.77e-06 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224205+0000", 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6429999999999998e-06 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224240+0000", 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5714999999999999e-05 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224555+0000", 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031487300000000002 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265911+0000", 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041355780000000002 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266040+0000", 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012856099999999999 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266158+0000", 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118521 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1636 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.631796344, 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042143574000000003, 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.882 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1636 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224731+0000", 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5600000000000001e-06 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224738+0000", 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224756+0000", 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8153999999999999e-05 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224770+0000", 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4168999999999999e-05 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266618+0000", 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041847662000000001 2024-09-14T07:08:30.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266730+0000", 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000111993 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266873+0000", 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014374400000000001 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1643 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 8978432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630715308999999, 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042103224000000002, 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1643 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225812+0000", 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.207e-06 2024-09-14T07:08:30.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225818+0000", 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225836+0000", 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8221999999999999e-05 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225849+0000", 2024-09-14T07:08:30.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.358e-05 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267705+0000", 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855723999999997 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267805+0000", 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9437000000000004e-05 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267914+0000", 2024-09-14T07:08:30.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109554 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1644 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630657081000001, 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042307838, 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1644 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225870+0000", 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2270000000000001e-06 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225876+0000", 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2569999999999998e-06 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225894+0000", 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8385000000000001e-05 2024-09-14T07:08:30.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225936+0000", 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2159000000000001e-05 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267952+0000", 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04201585 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268067+0000", 2024-09-14T07:08:30.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114985 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268177+0000", 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109975 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1645 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630568704, 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042484558999999998, 2024-09-14T07:08:30.895 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1645 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225959+0000", 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225978+0000", 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9466e-05 2024-09-14T07:08:30.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225998+0000", 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9315999999999999e-05 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226013+0000", 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268213+0000", 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200410000000001 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268324+0000", 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011123799999999999 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268442+0000", 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117889 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1646 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630492326000001, 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042595283999999997, 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1646 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226035+0000", 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5060000000000001e-06 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226042+0000", 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4370000000000004e-06 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226060+0000", 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8266000000000001e-05 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226074+0000", 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4029000000000001e-05 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042405023999999999 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268551+0000", 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1973999999999999e-05 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268629+0000", 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8047999999999995e-05 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.903 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1647 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12910592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630434292, 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042800796000000002, 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.904 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1647 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226093+0000", 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.46e-06 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226099+0000", 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3530000000000002e-06 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226118+0000", 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9055999999999998e-05 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226132+0000", 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268673+0000", 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042541272999999998 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268775+0000", 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102302 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268893+0000", 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117539 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1648 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630375150999999, 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042904468000000001, 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1648 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226152+0000", 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.234e-06 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226157+0000", 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2000000000000002e-06 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226175+0000", 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7059e-05 2024-09-14T07:08:30.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226204+0000", 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9536e-05 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268938+0000", 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042733698000000001 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268992+0000", 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4693e-05 2024-09-14T07:08:30.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269056+0000", 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3047999999999997e-05 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1649 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630300943, 2024-09-14T07:08:30.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043088794999999999, 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1649 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226227+0000", 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.33e-06 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226232+0000", 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.733e-06 2024-09-14T07:08:30.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226251+0000", 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8816000000000001e-05 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226264+0000", 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-05 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269097+0000", 2024-09-14T07:08:30.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0428324 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269197+0000", 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100419 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269314+0000", 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116897 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1651 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630242095, 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043199128000000003, 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1651 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.917 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226285+0000", 2024-09-14T07:08:30.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.237e-06 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226291+0000", 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1789999999999999e-06 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226311+0000", 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0656999999999999e-05 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226327+0000", 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5948999999999999e-05 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269352+0000", 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043024830999999999 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269419+0000", 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7281999999999997e-05 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269483+0000", 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3992999999999995e-05 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1700 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 32768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630178767, 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043417208999999998, 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1700 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:30.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226349+0000", 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3629999999999999e-06 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226354+0000", 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226372+0000", 2024-09-14T07:08:30.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7612999999999999e-05 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226386+0000", 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4146e-05 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269525+0000", 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043139380999999997 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269631+0000", 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105574 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269765+0000", 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133759 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1702 3.13 3:cfeabdd5:::rbd_data.11616dafc67f.000000000000000f:9 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:30.925 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630090346999999, 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043560261000000003, 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1702 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226437+0000", 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.567e-06 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226457+0000", 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395999999999999e-05 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226477+0000", 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9998999999999999e-05 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226491+0000", 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4503e-05 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269835+0000", 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043343995000000003 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269914+0000", 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9018999999999997e-05 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269996+0000", 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1781999999999997e-05 2024-09-14T07:08:30.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1740 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.630013453, 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043673828999999997, 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.930 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1740 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226514+0000", 2024-09-14T07:08:30.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4869999999999999e-06 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226520+0000", 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.976e-06 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226538+0000", 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7972999999999999e-05 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226551+0000", 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3142999999999999e-05 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270041+0000", 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043490174999999999 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270101+0000", 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9515999999999997e-05 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270187+0000", 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5558999999999999e-05 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.565239192, 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:30.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:30.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4803.0:1903 3.1f e43/21 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head v 43'21, mlcod=42'19)", 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.548078372999999, 2024-09-14T07:08:30.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.057064131999999997, 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308449+0000", 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.279e-06 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324156+0000", 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015707199000000002 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324195+0000", 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9070999999999999e-05 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324217+0000", 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1752000000000001e-05 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324333+0000", 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011607000000000001 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324359+0000", 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6296e-05 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365477+0000", 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041117065000000001 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365512+0000", 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-05 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.943 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4703.0:530 3.f e46/21 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head v 46'75, mlcod=45'73)", 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.9941394280000004, 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.075870370000000006, 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:30.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862389+0000", 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6579999999999998e-06 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870883+0000", 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084939100000000003 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870916+0000", 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2428000000000002e-05 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870931+0000", 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5656000000000001e-05 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871022+0000", 2024-09-14T07:08:30.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1063000000000005e-05 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871047+0000", 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4125e-05 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938212+0000", 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067164920000000003 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938257+0000", 2024-09-14T07:08:30.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5609999999999999e-05 2024-09-14T07:08:30.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.948 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:30.948 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:30.948 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:30.950 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:30.950 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:30.945+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:31.050 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:31.045+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:31.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:08:31.134 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:31.134 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:31.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "MOSDPGPush(3.3 55/54 [PushOp(3:c712f81b:::rbd_data.11615839fdc9.000000000000000b:head, version: 45'24, data_included: [0~581632,589824~122880,753664~98304,917504~65536,1048576~65536,1310720~32768,1376256~65536,1507328~16384], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:c712f81b:::rbd_data.11615839fdc9.000000000000000b:head@45'24, size: 7471104, copy_subset: [(0, 7471104)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1523712, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:31.133518+0000", 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0013077379999999999, 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001408096, 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.133518+0000", 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.133518+0000", 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.133518+0000", 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6000000000000004e-07 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.134159+0000", 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00064116100000000001 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.134184+0000", 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4995e-05 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.134207+0000", 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2515e-05 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.134317+0000", 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110121 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.134333+0000", 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.562e-05 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:08:31.138 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:31.151 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:31.149+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:31.154 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:31.253 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:31.249+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:31.332 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:31.332 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:31.346 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:31.354 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:31.349+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:31.455 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:31.453+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:31.470 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1404 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.251275088, 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044358094000000001, 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1404 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216853+0000", 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216859+0000", 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2669999999999997e-06 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216877+0000", 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7866000000000001e-05 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216891+0000", 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4712000000000001e-05 2024-09-14T07:08:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260962+0000", 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070703000000003 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261081+0000", 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011867 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261210+0000", 2024-09-14T07:08:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012960599999999999 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1407 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.251217343, 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044521318999999997, 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1407 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216911+0000", 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.333e-06 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216916+0000", 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0300000000000001e-06 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216933+0000", 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6616000000000002e-05 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216947+0000", 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3923e-05 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261260+0000", 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044313105999999998 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261323+0000", 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2688999999999996e-05 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261431+0000", 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108622 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1408 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.251161828000001, 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044764117999999999, 2024-09-14T07:08:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1408 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216967+0000", 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4360000000000001e-06 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216987+0000", 2024-09-14T07:08:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9689000000000001e-05 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217005+0000", 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8260000000000001e-05 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217018+0000", 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3396e-05 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261500+0000", 2024-09-14T07:08:31.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044481494000000003 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261601+0000", 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101755 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261730+0000", 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128088 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1561 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.246578940999999, 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044533453000000001, 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1561 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.484 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221569+0000", 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-05 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221589+0000", 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9266000000000002e-05 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221602+0000", 2024-09-14T07:08:31.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.366e-05 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265854+0000", 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044251429000000002 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265962+0000", 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266082+0000", 2024-09-14T07:08:31.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011999500000000001 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1562 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.246503014, 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044744586000000003, 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1562 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221625+0000", 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221631+0000", 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4729999999999999e-06 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221648+0000", 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7346e-05 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221663+0000", 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266141+0000", 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044478651000000001 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266249+0000", 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107371 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266369+0000", 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120258 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1563 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.246445439, 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044928442999999998, 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.492 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1563 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:31.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221683+0000", 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.55e-06 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221689+0000", 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1699999999999996e-06 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221706+0000", 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7286e-05 2024-09-14T07:08:31.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221719+0000", 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2959000000000001e-05 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266405+0000", 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044686664000000001 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266488+0000", 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2243999999999994e-05 2024-09-14T07:08:31.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266610+0000", 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012256999999999999 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1699 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.243910608, 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044262176, 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.496 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1699 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224218+0000", 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224254+0000", 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6525e-05 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224274+0000", 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0049999999999999e-05 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224289+0000", 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268246+0000", 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043957152999999999 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268354+0000", 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107433 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012504200000000001 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1701 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:31.500 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.243814658, 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044338850999999999, 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1701 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224314+0000", 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224320+0000", 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224339+0000", 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8592000000000001e-05 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224353+0000", 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.432e-05 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268522+0000", 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044169491999999998 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268578+0000", 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5099999999999998e-05 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268651+0000", 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-05 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1703 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:31.504 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.243750975999999, 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044549686999999998, 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1703 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224378+0000", 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.573e-06 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224384+0000", 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224403+0000", 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9113000000000001e-05 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224417+0000", 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4049000000000001e-05 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268700+0000", 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044282555000000001 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268805+0000", 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104858 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268926+0000", 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121296 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1704 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9764864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.243689635000001, 2024-09-14T07:08:31.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044649934000000002, 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224439+0000", 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224902+0000", 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046329699999999999 2024-09-14T07:08:31.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224924+0000", 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2059e-05 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224941+0000", 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835999999999999e-05 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268962+0000", 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044020653 2024-09-14T07:08:31.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269012+0000", 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0312000000000002e-05 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269088+0000", 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5457000000000006e-05 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.512 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1705 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.243157201000001, 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044393046999999998, 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224972+0000", 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.784e-06 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224979+0000", 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224996+0000", 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225010+0000", 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3926e-05 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269132+0000", 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044121736000000002 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269242+0000", 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010979700000000001 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269363+0000", 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121155 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.516 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1706 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.243098038999999, 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044499060999999999, 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225030+0000", 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225036+0000", 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4530000000000001e-06 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225053+0000", 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6983e-05 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225066+0000", 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269399+0000", 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044332900000000001 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269449+0000", 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9960000000000003e-05 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269528+0000", 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9221999999999995e-05 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1707 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.521 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.243043468, 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044705944999999997, 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225085+0000", 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225090+0000", 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.113e-06 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225107+0000", 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6569999999999999e-05 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225135+0000", 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8524999999999999e-05 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269571+0000", 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044435781000000001 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269676+0000", 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105374 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269790+0000", 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113415 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1708 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.242972206999999, 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044874641, 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225156+0000", 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225162+0000", 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5960000000000003e-06 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225179+0000", 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7235999999999999e-05 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225192+0000", 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269839+0000", 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044646439000000003 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269926+0000", 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7303000000000001e-05 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270030+0000", 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001036 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.529 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1709 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.242916089, 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045094769, 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.530 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225213+0000", 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6339999999999999e-06 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225218+0000", 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3959999999999997e-06 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225236+0000", 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8053000000000001e-05 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225250+0000", 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3583e-05 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270081+0000", 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044830868000000003 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270179+0000", 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8645000000000002e-05 2024-09-14T07:08:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270306+0000", 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012658999999999999 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1710 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.242858514, 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045251099000000003, 2024-09-14T07:08:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225270+0000", 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.173e-06 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225276+0000", 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6840000000000001e-06 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225292+0000", 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6722e-05 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225306+0000", 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3456e-05 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270348+0000", 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045042722 2024-09-14T07:08:31.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270431+0000", 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2182999999999996e-05 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270520+0000", 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9159000000000005e-05 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.538 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.171258041, 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:08:31.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:08:31.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:08:31.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.311782454, 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:08:31.544 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:08:31.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:08:31.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.311680169, 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:08:31.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:08:31.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.612628328, 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:08:31.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:31.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:08:31.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:08:31.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:31.558 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:31.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:31.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:31.561 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:31.557+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:31.661 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:31.657+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:31.713 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:31.713 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:31.729 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:31.729 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:31.730 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:31.730 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:31.731 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:31.731 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:31.744 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:31.747 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:31.747 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:31.761 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:31.757+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:31.766 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:31.766 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:31.785 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:31.785 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:31.804 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:31.804 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:31.861 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:31.861 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:31.863 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:31.861+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:31.874 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:31.963 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:31.961+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:31.997 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:31.997 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:32.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:32.063 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:32.061+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:08:32.158 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:32.158 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:32.163 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:32.161+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:32.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:32.265 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:32.261+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:32.342 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:32.342 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:32.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:32.403 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:32.401+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:32.466 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:32.461+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:32.488 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:32.488 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:32.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:32.567 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:32.561+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:32.668 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:32.665+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:32.675 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:32.675 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:32.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:32.775 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:32.773+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:32.861 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:32.861 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:32.869 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:32.865+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:32.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:32.970 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:32.965+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:33.050 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:33.051 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:33.065 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:33.111 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:33.109+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:33.171 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:33.169+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:33.193 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:33.193 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:33.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:33.273 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:33.269+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:33.374 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:33.369+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:33.383 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:33.383 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:33.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:33.474 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:33.469+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:33.520 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:33.520 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:33.535 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:33.579 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:33.577+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:33.665 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:33.665 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:33.676 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:33.673+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:33.679 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:33.776 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:33.773+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:33.801 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:33.801 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:33.815 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:33.822 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:33.877 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:33.873+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:33.977 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:33.973+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:33.978 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:33.978 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:33.985 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:33.985 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:33.995 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:34.000 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:34.012 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:34.012 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:34.032 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:34.050 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:34.051 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:34.068 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:34.087 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:34.085+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:34.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:08:34.169 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:34.169 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:34.169 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:34.169 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:34.180 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:34.177+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:34.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:34.281 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:34.277+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:34.352 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:34.353 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:34.366 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:34.382 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:34.377+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:34.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:08:34.465+0000 7fe7a5b68640 0 --2- 172.21.15.5:0/1749575430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78c0053d0 0x7fe78c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T07:08:34.483 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:34.481+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:34.534 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:34.534 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1404 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.315417246999999, 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044358094000000001, 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.535 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1404 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216853+0000", 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216859+0000", 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2669999999999997e-06 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216877+0000", 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7866000000000001e-05 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216891+0000", 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4712000000000001e-05 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260962+0000", 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070703000000003 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261081+0000", 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011867 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261210+0000", 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012960599999999999 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.539 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1407 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.315359502, 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044521318999999997, 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1407 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216911+0000", 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.333e-06 2024-09-14T07:08:34.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216916+0000", 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0300000000000001e-06 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216933+0000", 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6616000000000002e-05 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216947+0000", 2024-09-14T07:08:34.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3923e-05 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261260+0000", 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044313105999999998 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261323+0000", 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2688999999999996e-05 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261431+0000", 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108622 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1408 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.315303987, 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044764117999999999, 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1408 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216967+0000", 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4360000000000001e-06 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216987+0000", 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9689000000000001e-05 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217005+0000", 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8260000000000001e-05 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217018+0000", 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3396e-05 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261500+0000", 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044481494000000003 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261601+0000", 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101755 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261730+0000", 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128088 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.548 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1561 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.310721099999999, 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044533453000000001, 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1561 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:08:34.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221569+0000", 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-05 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221589+0000", 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9266000000000002e-05 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221602+0000", 2024-09-14T07:08:34.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.366e-05 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265854+0000", 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044251429000000002 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265962+0000", 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266082+0000", 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011999500000000001 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1562 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.310645173000001, 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044744586000000003, 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1562 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221625+0000", 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221631+0000", 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4729999999999999e-06 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221648+0000", 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7346e-05 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221663+0000", 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266141+0000", 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044478651000000001 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266249+0000", 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107371 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266369+0000", 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120258 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.557 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1563 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.310587598000001, 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044928442999999998, 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1563 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221683+0000", 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.55e-06 2024-09-14T07:08:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221689+0000", 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1699999999999996e-06 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221706+0000", 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7286e-05 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221719+0000", 2024-09-14T07:08:34.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2959000000000001e-05 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266405+0000", 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044686664000000001 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266488+0000", 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2243999999999994e-05 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266610+0000", 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012256999999999999 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1699 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.308052767, 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044262176, 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1699 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224218+0000", 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224254+0000", 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6525e-05 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224274+0000", 2024-09-14T07:08:34.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0049999999999999e-05 2024-09-14T07:08:34.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224289+0000", 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268246+0000", 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043957152999999999 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268354+0000", 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107433 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012504200000000001 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1701 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307956817000001, 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044338850999999999, 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.567 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1701 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224314+0000", 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224320+0000", 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224339+0000", 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8592000000000001e-05 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224353+0000", 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.432e-05 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268522+0000", 2024-09-14T07:08:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044169491999999998 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268578+0000", 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5099999999999998e-05 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268651+0000", 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-05 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1703 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307893135, 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044549686999999998, 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1703 2024-09-14T07:08:34.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224378+0000", 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.573e-06 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224384+0000", 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224403+0000", 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9113000000000001e-05 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224417+0000", 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4049000000000001e-05 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268700+0000", 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044282555000000001 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268805+0000", 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104858 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268926+0000", 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121296 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1704 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9764864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307831793999998, 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044649934000000002, 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.576 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224439+0000", 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224902+0000", 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046329699999999999 2024-09-14T07:08:34.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224924+0000", 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2059e-05 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224941+0000", 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835999999999999e-05 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268962+0000", 2024-09-14T07:08:34.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044020653 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269012+0000", 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0312000000000002e-05 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269088+0000", 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5457000000000006e-05 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1705 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307299360000002, 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044393046999999998, 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:08:34.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224972+0000", 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.784e-06 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224979+0000", 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224996+0000", 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225010+0000", 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3926e-05 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269132+0000", 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044121736000000002 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269242+0000", 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010979700000000001 2024-09-14T07:08:34.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269363+0000", 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121155 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1706 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307240197999999, 2024-09-14T07:08:34.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044499060999999999, 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225030+0000", 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225036+0000", 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4530000000000001e-06 2024-09-14T07:08:34.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225053+0000", 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6983e-05 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225066+0000", 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269399+0000", 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044332900000000001 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269449+0000", 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9960000000000003e-05 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269528+0000", 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9221999999999995e-05 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1707 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307185626999999, 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044705944999999997, 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:34.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225085+0000", 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225090+0000", 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.113e-06 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225107+0000", 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6569999999999999e-05 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225135+0000", 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8524999999999999e-05 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269571+0000", 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044435781000000001 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269676+0000", 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105374 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269790+0000", 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113415 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1708 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.594 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307114366, 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044874641, 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225156+0000", 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225162+0000", 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5960000000000003e-06 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225179+0000", 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7235999999999999e-05 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225192+0000", 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269839+0000", 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044646439000000003 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269926+0000", 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7303000000000001e-05 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270030+0000", 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001036 2024-09-14T07:08:34.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1709 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307058248000001, 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045094769, 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225213+0000", 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6339999999999999e-06 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225218+0000", 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3959999999999997e-06 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225236+0000", 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8053000000000001e-05 2024-09-14T07:08:34.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225250+0000", 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3583e-05 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270081+0000", 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044830868000000003 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270179+0000", 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8645000000000002e-05 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270306+0000", 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012658999999999999 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1710 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:34.603 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.307000673000001, 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045251099000000003, 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225270+0000", 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.173e-06 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225276+0000", 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6840000000000001e-06 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225292+0000", 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6722e-05 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225306+0000", 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3456e-05 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270348+0000", 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045042722 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270431+0000", 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2182999999999996e-05 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270520+0000", 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9159000000000005e-05 2024-09-14T07:08:34.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.235400200000001, 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:34.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:08:34.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:08:34.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:34.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3759246130000005, 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.614 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.618 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3758223279999999, 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:08:34.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:08:34.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.6767704869999998, 2024-09-14T07:08:34.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:08:34.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:34.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:34.628 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:34.630 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:34.630 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:34.625+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:34.731 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:34.729+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:34.801 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:34.802 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:34.802 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:34.802 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:34.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:34.832 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:34.829+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:34.933 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:34.929+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:34.939 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:34.939 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:34.953 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:35.033 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:35.029+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.245475145, 2024-09-14T07:08:35.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:35.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.244700797, 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.088 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:35.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.092 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3038 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946478623000001, 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046439165999999997, 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3038 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129535+0000", 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3230000000000001e-06 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129541+0000", 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.054e-06 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129559+0000", 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8329e-05 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129573+0000", 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175474+0000", 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045901019000000001 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175554+0000", 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0661000000000006e-05 2024-09-14T07:08:35.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175973+0000", 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041842300000000002 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3039 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946420943, 2024-09-14T07:08:35.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046967035999999997, 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3039 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129592+0000", 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.206e-06 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129598+0000", 2024-09-14T07:08:35.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2340000000000002e-06 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129616+0000", 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8003e-05 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129629+0000", 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2913e-05 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176018+0000", 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046389224999999999 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176108+0000", 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176558+0000", 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044997100000000002 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3040 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946331741000002, 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047451337000000003, 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3040 2024-09-14T07:08:35.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129682+0000", 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.607e-06 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129688+0000", 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129708+0000", 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0047e-05 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129723+0000", 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.502e-05 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176597+0000", 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046873259 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176684+0000", 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7349999999999995e-05 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177132+0000", 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044784099999999998 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3041 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946268438000001, 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047622767000000003, 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3041 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129745+0000", 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129751+0000", 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129773+0000", 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782999999999999e-05 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129788+0000", 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4606999999999999e-05 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177167+0000", 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047378941000000001 2024-09-14T07:08:35.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177253+0000", 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5704999999999993e-05 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177367+0000", 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113861 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3042 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946204998999999, 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048188976000000001, 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3042 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129808+0000", 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:35.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129814+0000", 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7699999999999998e-06 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129832+0000", 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.751e-05 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129844+0000", 2024-09-14T07:08:35.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2797e-05 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177436+0000", 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047591574999999997 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177545+0000", 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109446 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177996+0000", 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045071800000000002 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3043 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946148545, 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048376965000000001, 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3043 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129865+0000", 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129870+0000", 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129888+0000", 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.755e-05 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129902+0000", 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178049+0000", 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048146648 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178131+0000", 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2619000000000003e-05 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178241+0000", 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109262 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.119 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3044 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946076466000001, 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048575449, 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3044 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129937+0000", 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1930000000000001e-06 2024-09-14T07:08:35.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129943+0000", 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129961+0000", 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8034e-05 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129975+0000", 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3696000000000001e-05 2024-09-14T07:08:35.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178293+0000", 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048318051000000001 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178392+0000", 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9038999999999998e-05 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178511+0000", 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119493 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3045 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.946017210000001, 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049079214000000003, 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3045 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:35.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129996+0000", 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1170000000000001e-06 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130002+0000", 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130024+0000", 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1763000000000002e-05 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130059+0000", 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.536e-05 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178555+0000", 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048495599 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178636+0000", 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1112000000000005e-05 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179074+0000", 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043820000000000003 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3046 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.128 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.945930533999999, 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049550174000000002, 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3046 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130083+0000", 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130089+0000", 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130108+0000", 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9046999999999999e-05 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130122+0000", 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4137e-05 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179126+0000", 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049003927000000003 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179221+0000", 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4717999999999997e-05 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179632+0000", 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041092199999999998 2024-09-14T07:08:35.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3047 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.945869837, 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050053944000000003, 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3047 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130144+0000", 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3030000000000001e-06 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130149+0000", 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6230000000000002e-06 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130169+0000", 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9386999999999999e-05 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130182+0000", 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3706000000000001e-05 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179677+0000", 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049494109000000001 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179763+0000", 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6473999999999999e-05 2024-09-14T07:08:35.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180196+0000", 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043334200000000003 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3048 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.945809917999998, 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050582655999999997, 2024-09-14T07:08:35.137 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3048 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130204+0000", 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.266e-06 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130219+0000", 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5549999999999999e-05 2024-09-14T07:08:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130235+0000", 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6030000000000001e-05 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130249+0000", 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3977e-05 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180264+0000", 2024-09-14T07:08:35.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050014851999999999 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180359+0000", 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5400000000000001e-05 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180785+0000", 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042558100000000002 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3049 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.945742452000001, 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.051156504999999998, 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3049 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130271+0000", 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130277+0000", 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6370000000000004e-06 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130297+0000", 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.997e-05 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130311+0000", 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4358999999999999e-05 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180833+0000", 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050521863 2024-09-14T07:08:35.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180936+0000", 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010296799999999999 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.181426+0000", 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049044799999999999 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3149 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.930555876, 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046930569999999998, 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3149 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145458+0000", 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:08:35.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145463+0000", 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6169999999999998e-06 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145486+0000", 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2626e-05 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145500+0000", 2024-09-14T07:08:35.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4243000000000001e-05 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191669+0000", 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046169090000000003 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191811+0000", 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014187100000000001 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192387+0000", 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057567299999999996 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3150 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.930489516000002, 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047680769999999997, 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3150 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:35.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145524+0000", 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.39e-06 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145647+0000", 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123242 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145673+0000", 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145711+0000", 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8745999999999998e-05 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192444+0000", 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046732462000000002 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192602+0000", 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015825400000000001 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193203+0000", 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00060150600000000002 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3172 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:35.155 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.930103645999999, 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048262865000000002, 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3172 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:35.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145911+0000", 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145920+0000", 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2969999999999992e-06 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145947+0000", 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7149999999999999e-05 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145965+0000", 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8295999999999999e-05 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193303+0000", 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047338247 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193492+0000", 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018843500000000001 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194171+0000", 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067951700000000003 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.159 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3173 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.93001829, 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048937952999999999, 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:35.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3173 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145995+0000", 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146021+0000", 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5684e-05 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146044+0000", 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2503000000000001e-05 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146059+0000", 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5043e-05 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194258+0000", 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04819975 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194372+0000", 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113373 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194932+0000", 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056004399999999999 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.233517798999999, 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.164 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:08:35.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:35.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:35.170 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.936589593000001, 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:35.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:08:35.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:35.176 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:35.176 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 2, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:35.176 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:35.177 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T07:08:35.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:08:35.179 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:08:35.179 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:35.173+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:35.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:35.279 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:35.277+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:35.352 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:35.352 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:35.353 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:35.353 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:35.370 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:35.415 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:35.413+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:35.480 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:35.477+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:35.532 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:35.532 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:35.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:08:35.547 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 59, 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:08:35.101230+0000", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:08:30.073135+0000", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5801984, 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "noscrub", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:08:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:08:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:08:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:08:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:08:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:08:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:08:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 6.059999942779541, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 6.059999942779541, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.33000001311302185, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:08:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:08:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:08:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:08:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:08:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:08:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 2.25, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2.25, 2024-09-14T07:08:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:08:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:08:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "51", 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:08:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 12, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 51, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:08:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:08:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:08:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.690000057220459, 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.690000057220459, 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.690000057220459, 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.690000057220459, 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:08:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.297698974609375, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 57, 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:08:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 56, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:08:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 56, 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 54, 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:08:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:08:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 57, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.595 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:08:35.597 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 57, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.598 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.599 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:08:35.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:08:35.602 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:35.603 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:08:35.604 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:08:35.605 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:35.606 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.607 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:08:35.608 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:08:35.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.610 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:35.612 INFO:tasks.thrashosds.thrasher:j is {'epoch': 59, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:08:35.101230+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:08:30.073135+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 12, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.297698974609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 54, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:08:35.612 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:08:35.613 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:35.609+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:35.712 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:35.709+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:35.741 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:368 3.1f 3:f8928d1a:::rbd_data.1161a9b4f57.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.675052175000001, 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01725641, 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 368 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:35.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064165+0000", 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.540000000000001e-07 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064170+0000", 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8940000000000001e-06 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064185+0000", 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5182e-05 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064193+0000", 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7999999999999999e-06 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064281+0000", 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7517999999999998e-05 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064365+0000", 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4728000000000006e-05 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:35.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064567+0000", 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201757 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068739+0000", 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041723519999999998 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081335+0000", 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012595492999999999 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081366+0000", 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1538999999999997e-05 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081421+0000", 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4193000000000001e-05 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.748 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:373 3.1f 3:fcd26671:::rbd_data.1161a9b4f57.0000000000000045:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.674699436000001, 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017078697, 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 373 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064518+0000", 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064524+0000", 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9839999999999998e-06 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064536+0000", 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2089999999999999e-05 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064544+0000", 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1589999999999993e-06 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065143+0000", 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059867500000000001 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065216+0000", 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3497000000000005e-05 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:35.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065416+0000", 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019953899999999999 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068744+0000", 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033285459999999999 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081472+0000", 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012727815 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081528+0000", 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5958000000000002e-05 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081596+0000", 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7489000000000004e-05 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:386 3.1f 3:fbac3711:::rbd_data.1161a9b4f57.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:35.754 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.672955593000001, 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015471182999999999, 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 386 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066262+0000", 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5600000000000004e-07 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066268+0000", 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0809999999999999e-06 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066279+0000", 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1389999999999999e-05 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066286+0000", 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8959999999999997e-06 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066393+0000", 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107413 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066451+0000", 2024-09-14T07:08:35.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7454000000000003e-05 2024-09-14T07:08:35.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:35.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066556+0000", 2024-09-14T07:08:35.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105326 2024-09-14T07:08:35.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:35.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068748+0000", 2024-09-14T07:08:35.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002192285 2024-09-14T07:08:35.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:35.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081641+0000", 2024-09-14T07:08:35.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012892280000000001 2024-09-14T07:08:35.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081688+0000", 2024-09-14T07:08:35.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-09-14T07:08:35.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081732+0000", 2024-09-14T07:08:35.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4132999999999999e-05 2024-09-14T07:08:35.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.763 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.764 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:286 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.764 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:35.764 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270980253000001, 2024-09-14T07:08:35.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01542496, 2024-09-14T07:08:35.764 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.765 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.765 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.765 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 286 2024-09-14T07:08:35.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.765 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:35.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:35.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468238+0000", 2024-09-14T07:08:35.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3960000000000001e-06 2024-09-14T07:08:35.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468244+0000", 2024-09-14T07:08:35.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:08:35.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468267+0000", 2024-09-14T07:08:35.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3603000000000001e-05 2024-09-14T07:08:35.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468278+0000", 2024-09-14T07:08:35.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0883000000000001e-05 2024-09-14T07:08:35.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483169+0000", 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014890970999999999 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483418+0000", 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024896800000000002 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483661+0000", 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024296499999999999 2024-09-14T07:08:35.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:287 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270923281000002, 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015987674, 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 287 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468294+0000", 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6300000000000004e-07 2024-09-14T07:08:35.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468329+0000", 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4799999999999999e-05 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468349+0000", 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0299999999999999e-05 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468363+0000", 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4284000000000001e-05 2024-09-14T07:08:35.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483821+0000", 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015458120000000001 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484068+0000", 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000246281 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484281+0000", 2024-09-14T07:08:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000213026 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:288 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270828581, 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016408030000000001, 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 288 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468389+0000", 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.64e-06 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468396+0000", 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468411+0000", 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5153e-05 2024-09-14T07:08:35.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468419+0000", 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7039999999999994e-06 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484366+0000", 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015946689999999999 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484617+0000", 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025165400000000003 2024-09-14T07:08:35.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484796+0000", 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017846299999999999 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:289 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 122880~6656] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270784601999999, 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111327999999999, 2024-09-14T07:08:35.781 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 289 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468433+0000", 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1999999999999999e-07 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468436+0000", 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3969999999999998e-06 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468445+0000", 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9800000000000004e-06 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468455+0000", 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6900000000000004e-06 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484948+0000", 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016493799 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485216+0000", 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026811499999999998 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485543+0000", 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032662700000000002 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.785 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:290 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270751555, 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017722122999999999, 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 290 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4000000000000001e-07 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468469+0000", 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7529999999999999e-06 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468478+0000", 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6799999999999999e-06 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468485+0000", 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.153e-06 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485735+0000", 2024-09-14T07:08:35.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017249970999999999 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485956+0000", 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022074799999999999 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486187+0000", 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023117799999999999 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:291 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270720479000001, 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018193804000000001, 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 291 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468497+0000", 2024-09-14T07:08:35.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468502+0000", 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5439999999999998e-06 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468531+0000", 2024-09-14T07:08:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8898999999999999e-05 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468543+0000", 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1756999999999999e-05 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486295+0000", 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017751423999999998 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486492+0000", 2024-09-14T07:08:35.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019711899999999999 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486690+0000", 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019808099999999999 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:292 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270613676, 2024-09-14T07:08:35.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018553717000000001, 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 292 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468604+0000", 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5999999999999999e-06 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468611+0000", 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7800000000000003e-06 2024-09-14T07:08:35.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468633+0000", 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1710000000000001e-05 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468647+0000", 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4209999999999999e-05 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486764+0000", 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018116538000000001 2024-09-14T07:08:35.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486948+0000", 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018412999999999999 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487156+0000", 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020874899999999999 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:293 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.270547113999999, 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019096847, 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 293 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468671+0000", 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468676+0000", 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.857e-06 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468694+0000", 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7629999999999999e-05 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468712+0000", 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7762999999999999e-05 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487288+0000", 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018576294 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487555+0000", 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026692399999999998 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487766+0000", 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021110899999999999 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.803 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.450475434000001, 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:35.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.810 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.420967803, 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:08:35.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:08:35.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.817 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.412320443999999, 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:08:35.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:08:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:08:35.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:08:35.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:341 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:b [sparse-read 9764864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.123910386, 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015588305, 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 341 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615307+0000", 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615312+0000", 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615331+0000", 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8542e-05 2024-09-14T07:08:35.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615346+0000", 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4826e-05 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.629124+0000", 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013778367999999999 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.629369+0000", 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000244842 2024-09-14T07:08:35.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.630894+0000", 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015252600000000001 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.746228104, 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:08:35.828 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.579498878000001, 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:35.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:08:35.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:08:35.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:08:35.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.397567364, 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:08:35.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6464978699999993, 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:08:35.841 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4732.0:183 3.1b 3:d96d4dd6:::rbd_data.1161bf39a192.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 6709248~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T07:08:35.846 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.7966278300000003, 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030801473999999999, 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4732", 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/303097868", 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 183 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:35.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942590+0000", 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-09-14T07:08:35.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944540+0000", 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001949824 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944564+0000", 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4199e-05 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944574+0000", 2024-09-14T07:08:35.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944644+0000", 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0240999999999994e-05 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944798+0000", 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153432 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.945082+0000", 2024-09-14T07:08:35.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028403899999999998 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.959401+0000", 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014319159999999999 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973083+0000", 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013682016 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973261+0000", 2024-09-14T07:08:35.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017752999999999999 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973390+0000", 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012949500000000001 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:35.852 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:35.853 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:35.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:35.855 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:35.853+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:35.955 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:35.949+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:35.961 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:08:35.961 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:08:35.979 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2024-09-14T07:08:35.220606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":700123343,"num_objects":355,"num_object_clones":50,"num_object_copies":710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":355,"num_whiteouts":120,"num_read":39520,"num_read_kb":198087,"num_write":3826,"num_write_kb":137978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":16104881,"num_keys_recovered":70,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2652,"ondisk_log_size":2652,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":231516,"kb_used_data":19396,"kb_used_omap":9,"kb_used_meta":211958,"kb_avail":565999524,"statfs":{"total":579820584960,"available":579583512576,"internally_reserved":0,"allocated":19861504,"data_stored":242774268,"data_compressed":4213882,"data_compressed_allocated":14331904,"data_compressed_original":240201728,"omap_allocated":9568,"internal_metadata":217045664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,6],"upper_bound":8},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":-1,"num_objects_missing":-1,"num_objects_degraded":0,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":7541248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000987"},"pg_stats":[{"pgid":"3.1f","version":"45'23","reported_seq":403,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.093363+0000","last_change":"2024-09-14T07:07:49.800921+0000","last_active":"2024-09-14T07:08:30.093363+0000","last_peered":"2024-09-14T07:08:30.093363+0000","last_clean":"2024-09-14T07:08:30.093363+0000","last_became_active":"2024-09-14T07:07:47.794275+0000","last_became_peered":"2024-09-14T07:07:47.794275+0000","last_unstale":"2024-09-14T07:08:30.093363+0000","last_undegraded":"2024-09-14T07:08:30.093363+0000","last_fullsized":"2024-09-14T07:08:30.093363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:02:42.364780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":15122448,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":308,"num_read_kb":12355,"num_write":96,"num_write_kb":7518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"45'257","reported_seq":1414,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.081466+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:08:30.081466+0000","last_peered":"2024-09-14T07:08:30.081466+0000","last_clean":"2024-09-14T07:08:30.081466+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:08:30.081466+0000","last_undegraded":"2024-09-14T07:08:30.081466+0000","last_fullsized":"2024-09-14T07:08:30.081466+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:29.205982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":36427962,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2182,"num_read_kb":18777,"num_write":488,"num_write_kb":8859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"41'191","reported_seq":3182,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:29.075451+0000","last_change":"2024-09-14T07:08:30.094208+0000","last_active":"2024-09-14T07:08:30.094208+0000","last_peered":"2024-09-14T07:08:29.075451+0000","last_clean":"2024-09-14T07:08:29.075451+0000","last_became_active":"2024-09-14T07:08:28.073844+0000","last_became_peered":"2024-09-14T07:08:28.073844+0000","last_unstale":"2024-09-14T07:08:29.075451+0000","last_undegraded":"2024-09-14T07:08:29.075451+0000","last_fullsized":"2024-09-14T07:08:29.075451+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":5660672,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":4346,"num_read_kb":4947,"num_write":97,"num_write_kb":287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":68096,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"52'45","reported_seq":909,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:29.075437+0000","last_change":"2024-09-14T07:08:30.094936+0000","last_active":"2024-09-14T07:08:30.094936+0000","last_peered":"2024-09-14T07:08:29.075437+0000","last_clean":"2024-09-14T07:08:29.075437+0000","last_became_active":"2024-09-14T07:08:25.058802+0000","last_became_peered":"2024-09-14T07:08:25.058802+0000","last_unstale":"2024-09-14T07:08:29.075437+0000","last_undegraded":"2024-09-14T07:08:29.075437+0000","last_fullsized":"2024-09-14T07:08:29.075437+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":13959168,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":771,"num_read_kb":4187,"num_write":47,"num_write_kb":1218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":890368,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"53'121","reported_seq":2004,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:30.067006+0000","last_change":"2024-09-14T07:08:30.093211+0000","last_active":"2024-09-14T07:08:30.093211+0000","last_peered":"2024-09-14T07:08:30.067006+0000","last_clean":"2024-09-14T07:08:30.067006+0000","last_became_active":"2024-09-14T07:08:28.074990+0000","last_became_peered":"2024-09-14T07:08:28.074990+0000","last_unstale":"2024-09-14T07:08:30.067006+0000","last_undegraded":"2024-09-14T07:08:30.067006+0000","last_fullsized":"2024-09-14T07:08:30.067006+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":51007124,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":1744,"num_read_kb":7313,"num_write":138,"num_write_kb":6858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":776340,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"45'109","reported_seq":2247,"reported_epoch":49,"state":"remapped+peering","last_fresh":"2024-09-14T07:08:25.052054+0000","last_change":"2024-09-14T07:08:25.052054+0000","last_active":"2024-09-14T07:08:24.034024+0000","last_peered":"2024-09-14T07:08:19.244919+0000","last_clean":"2024-09-14T07:08:19.244919+0000","last_became_active":"2024-09-14T07:07:47.794385+0000","last_became_peered":"2024-09-14T07:07:47.794385+0000","last_unstale":"2024-09-14T07:08:25.052054+0000","last_undegraded":"2024-09-14T07:08:25.052054+0000","last_fullsized":"2024-09-14T07:08:25.052054+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":43253779,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":3286,"num_read_kb":6031,"num_write":142,"num_write_kb":8388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"42'80","reported_seq":900,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:29.080030+0000","last_change":"2024-09-14T07:08:30.087250+0000","last_active":"2024-09-14T07:08:30.087250+0000","last_peered":"2024-09-14T07:08:29.080030+0000","last_clean":"2024-09-14T07:08:29.080030+0000","last_became_active":"2024-09-14T07:08:25.058068+0000","last_became_peered":"2024-09-14T07:08:25.058068+0000","last_unstale":"2024-09-14T07:08:29.080030+0000","last_undegraded":"2024-09-14T07:08:29.080030+0000","last_fullsized":"2024-09-14T07:08:29.080030+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":17249830,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":879,"num_read_kb":4055,"num_write":97,"num_write_kb":1138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1061414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"50'83","reported_seq":877,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.081339+0000","last_change":"2024-09-14T07:07:53.859978+0000","last_active":"2024-09-14T07:08:30.081339+0000","last_peered":"2024-09-14T07:08:30.081339+0000","last_clean":"2024-09-14T07:08:30.081339+0000","last_became_active":"2024-09-14T07:07:53.840766+0000","last_became_peered":"2024-09-14T07:07:53.840766+0000","last_unstale":"2024-09-14T07:08:30.081339+0000","last_undegraded":"2024-09-14T07:08:30.081339+0000","last_fullsized":"2024-09-14T07:08:30.081339+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:48:46.878678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":46902320,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":877,"num_read_kb":3221,"num_write":207,"num_write_kb":12777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"45'25","reported_seq":197,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.094341+0000","last_change":"2024-09-14T07:08:19.008803+0000","last_active":"2024-09-14T07:08:30.094341+0000","last_peered":"2024-09-14T07:08:30.094341+0000","last_clean":"2024-09-14T07:08:30.094341+0000","last_became_active":"2024-09-14T07:08:19.007743+0000","last_became_peered":"2024-09-14T07:08:19.007743+0000","last_unstale":"2024-09-14T07:08:30.094341+0000","last_undegraded":"2024-09-14T07:08:30.094341+0000","last_fullsized":"2024-09-14T07:08:30.094341+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:38:06.387063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":1462310,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":56,"num_read_kb":301,"num_write":32,"num_write_kb":656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"45'41","reported_seq":1342,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.095549+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:08:30.095549+0000","last_peered":"2024-09-14T07:08:30.095549+0000","last_clean":"2024-09-14T07:08:30.095549+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:08:30.095549+0000","last_undegraded":"2024-09-14T07:08:30.095549+0000","last_fullsized":"2024-09-14T07:08:30.095549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:22.849098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":29158400,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":1233,"num_read_kb":9795,"num_write":30,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"50'80","reported_seq":1122,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.083450+0000","last_change":"2024-09-14T07:07:49.804465+0000","last_active":"2024-09-14T07:08:30.083450+0000","last_peered":"2024-09-14T07:08:30.083450+0000","last_clean":"2024-09-14T07:08:30.083450+0000","last_became_active":"2024-09-14T07:07:47.804440+0000","last_became_peered":"2024-09-14T07:07:47.804440+0000","last_unstale":"2024-09-14T07:08:30.083450+0000","last_undegraded":"2024-09-14T07:08:30.083450+0000","last_fullsized":"2024-09-14T07:08:30.083450+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:11.537222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":20339761,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1269,"num_read_kb":16521,"num_write":163,"num_write_kb":11798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"46'68","reported_seq":1312,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:29.070457+0000","last_change":"2024-09-14T07:08:30.085083+0000","last_active":"2024-09-14T07:08:30.085083+0000","last_peered":"2024-09-14T07:08:29.070457+0000","last_clean":"2024-09-14T07:08:29.070457+0000","last_became_active":"2024-09-14T07:08:25.051237+0000","last_became_peered":"2024-09-14T07:08:25.051237+0000","last_unstale":"2024-09-14T07:08:29.070457+0000","last_undegraded":"2024-09-14T07:08:29.070457+0000","last_fullsized":"2024-09-14T07:08:29.070457+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":12910096,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":7,"num_read":1576,"num_read_kb":4144,"num_write":38,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":202256,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":95,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:29.071312+0000","last_change":"2024-09-14T07:08:30.084044+0000","last_active":"2024-09-14T07:08:30.084044+0000","last_peered":"2024-09-14T07:08:29.071312+0000","last_clean":"2024-09-14T07:08:29.071312+0000","last_became_active":"2024-09-14T07:08:25.047025+0000","last_became_peered":"2024-09-14T07:08:25.047025+0000","last_unstale":"2024-09-14T07:08:29.071312+0000","last_undegraded":"2024-09-14T07:08:29.071312+0000","last_fullsized":"2024-09-14T07:08:29.071312+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"51'30","reported_seq":385,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.092900+0000","last_change":"2024-09-14T07:07:49.801738+0000","last_active":"2024-09-14T07:08:30.092900+0000","last_peered":"2024-09-14T07:08:30.092900+0000","last_clean":"2024-09-14T07:08:30.092900+0000","last_became_active":"2024-09-14T07:07:47.793157+0000","last_became_peered":"2024-09-14T07:07:47.793157+0000","last_unstale":"2024-09-14T07:08:30.092900+0000","last_undegraded":"2024-09-14T07:08:30.092900+0000","last_fullsized":"2024-09-14T07:08:30.092900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:32.307138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":129664,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":327,"num_read_kb":362,"num_write":34,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.087926+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:08:30.087926+0000","last_peered":"2024-09-14T07:08:30.087926+0000","last_clean":"2024-09-14T07:08:30.087926+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:08:30.087926+0000","last_undegraded":"2024-09-14T07:08:30.087926+0000","last_fullsized":"2024-09-14T07:08:30.087926+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"53'47","reported_seq":539,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.081510+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:08:30.081510+0000","last_peered":"2024-09-14T07:08:30.081510+0000","last_clean":"2024-09-14T07:08:30.081510+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:08:30.081510+0000","last_undegraded":"2024-09-14T07:08:30.081510+0000","last_fullsized":"2024-09-14T07:08:30.081510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:57:34.979161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":5703717,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":786,"num_read_kb":2467,"num_write":39,"num_write_kb":232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.085640+0000","last_change":"2024-09-14T07:07:38.031148+0000","last_active":"2024-09-14T07:08:30.085640+0000","last_peered":"2024-09-14T07:08:30.085640+0000","last_clean":"2024-09-14T07:08:30.085640+0000","last_became_active":"2024-09-14T07:07:36.196549+0000","last_became_peered":"2024-09-14T07:07:36.196549+0000","last_unstale":"2024-09-14T07:08:30.085640+0000","last_undegraded":"2024-09-14T07:08:30.085640+0000","last_fullsized":"2024-09-14T07:08:30.085640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:35.911258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"43'90","reported_seq":1615,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.087569+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:08:30.087569+0000","last_peered":"2024-09-14T07:08:30.087569+0000","last_clean":"2024-09-14T07:08:30.087569+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:08:30.087569+0000","last_undegraded":"2024-09-14T07:08:30.087569+0000","last_fullsized":"2024-09-14T07:08:30.087569+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:29:51.109157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":262144,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":2183,"num_read_kb":2037,"num_write":51,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.087676+0000","last_change":"2024-09-14T07:07:38.032270+0000","last_active":"2024-09-14T07:08:30.087676+0000","last_peered":"2024-09-14T07:08:30.087676+0000","last_clean":"2024-09-14T07:08:30.087676+0000","last_became_active":"2024-09-14T07:07:36.196923+0000","last_became_peered":"2024-09-14T07:07:36.196923+0000","last_unstale":"2024-09-14T07:08:30.087676+0000","last_undegraded":"2024-09-14T07:08:30.087676+0000","last_fullsized":"2024-09-14T07:08:30.087676+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:38:20.173745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"40'32","reported_seq":1231,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.094895+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:08:30.094895+0000","last_peered":"2024-09-14T07:08:30.094895+0000","last_clean":"2024-09-14T07:08:30.094895+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:08:30.094895+0000","last_undegraded":"2024-09-14T07:08:30.094895+0000","last_fullsized":"2024-09-14T07:08:30.094895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:59.802313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":7139360,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":1032,"num_read_kb":7347,"num_write":42,"num_write_kb":1900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"45'24","reported_seq":41,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:08:32.098654+0000","last_change":"2024-09-14T07:08:31.236451+0000","last_active":"2024-09-14T07:08:32.098654+0000","last_peered":"2024-09-14T07:08:32.098654+0000","last_clean":"2024-09-14T07:08:32.098654+0000","last_became_active":"2024-09-14T07:08:31.107164+0000","last_became_peered":"2024-09-14T07:08:31.107164+0000","last_unstale":"2024-09-14T07:08:32.098654+0000","last_undegraded":"2024-09-14T07:08:32.098654+0000","last_fullsized":"2024-09-14T07:08:32.098654+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:03:28.236628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":28934144,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":94,"num_read_kb":1575,"num_write":94,"num_write_kb":7461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":11512320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.094948+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:08:30.094948+0000","last_peered":"2024-09-14T07:08:30.094948+0000","last_clean":"2024-09-14T07:08:30.094948+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:08:30.094948+0000","last_undegraded":"2024-09-14T07:08:30.094948+0000","last_fullsized":"2024-09-14T07:08:30.094948+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"45'22","reported_seq":351,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.086842+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:08:30.086842+0000","last_peered":"2024-09-14T07:08:30.086842+0000","last_clean":"2024-09-14T07:08:30.086842+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:08:30.086842+0000","last_undegraded":"2024-09-14T07:08:30.086842+0000","last_fullsized":"2024-09-14T07:08:30.086842+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:16:44.683720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":1212578,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":274,"num_read_kb":1692,"num_write":26,"num_write_kb":1121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.082114+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:08:30.082114+0000","last_peered":"2024-09-14T07:08:30.082114+0000","last_clean":"2024-09-14T07:08:30.082114+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:08:30.082114+0000","last_undegraded":"2024-09-14T07:08:30.082114+0000","last_fullsized":"2024-09-14T07:08:30.082114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"42'48","reported_seq":55,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:08:32.098748+0000","last_change":"2024-09-14T07:08:31.263248+0000","last_active":"2024-09-14T07:08:32.098748+0000","last_peered":"2024-09-14T07:08:32.098748+0000","last_clean":"2024-09-14T07:08:32.098748+0000","last_became_active":"2024-09-14T07:08:31.108303+0000","last_became_peered":"2024-09-14T07:08:31.108303+0000","last_unstale":"2024-09-14T07:08:32.098748+0000","last_undegraded":"2024-09-14T07:08:32.098748+0000","last_fullsized":"2024-09-14T07:08:32.098748+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:24:30.630966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":491561,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":204,"num_read_kb":804,"num_write":49,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":14377,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.083007+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:08:30.083007+0000","last_peered":"2024-09-14T07:08:30.083007+0000","last_clean":"2024-09-14T07:08:30.083007+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:08:30.083007+0000","last_undegraded":"2024-09-14T07:08:30.083007+0000","last_fullsized":"2024-09-14T07:08:30.083007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"53'54","reported_seq":849,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.087631+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:08:30.087631+0000","last_peered":"2024-09-14T07:08:30.087631+0000","last_clean":"2024-09-14T07:08:30.087631+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:08:30.087631+0000","last_undegraded":"2024-09-14T07:08:30.087631+0000","last_fullsized":"2024-09-14T07:08:30.087631+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:14:07.558653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":34824720,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":728,"num_read_kb":1283,"num_write":48,"num_write_kb":592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":111,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.094508+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:08:30.094508+0000","last_peered":"2024-09-14T07:08:30.094508+0000","last_clean":"2024-09-14T07:08:30.094508+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:08:30.094508+0000","last_undegraded":"2024-09-14T07:08:30.094508+0000","last_fullsized":"2024-09-14T07:08:30.094508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":181,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.083043+0000","last_change":"2024-09-14T07:07:32.604869+0000","last_active":"2024-09-14T07:08:30.083043+0000","last_peered":"2024-09-14T07:08:30.083043+0000","last_clean":"2024-09-14T07:08:30.083043+0000","last_became_active":"2024-09-14T07:07:32.602241+0000","last_became_peered":"2024-09-14T07:07:32.602241+0000","last_unstale":"2024-09-14T07:08:30.083043+0000","last_undegraded":"2024-09-14T07:08:30.083043+0000","last_fullsized":"2024-09-14T07:08:30.083043+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:25.905190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"50'87","reported_seq":3745,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.081287+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:08:30.081287+0000","last_peered":"2024-09-14T07:08:30.081287+0000","last_clean":"2024-09-14T07:08:30.081287+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:08:30.081287+0000","last_undegraded":"2024-09-14T07:08:30.081287+0000","last_fullsized":"2024-09-14T07:08:30.081287+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:50:52.542911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":59524111,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":3471,"num_read_kb":28069,"num_write":243,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"45'181","reported_seq":403,"reported_epoch":49,"state":"remapped+peering","last_fresh":"2024-09-14T07:08:25.044207+0000","last_change":"2024-09-14T07:08:25.044207+0000","last_active":"2024-09-14T07:08:24.036204+0000","last_peered":"2024-09-14T07:08:19.215962+0000","last_clean":"2024-09-14T07:08:19.215962+0000","last_became_active":"2024-09-14T07:07:53.839838+0000","last_became_peered":"2024-09-14T07:07:53.839838+0000","last_unstale":"2024-09-14T07:08:25.044207+0000","last_undegraded":"2024-09-14T07:08:25.044207+0000","last_fullsized":"2024-09-14T07:08:25.044207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":29577448,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":871,"num_read_kb":1891,"num_write":270,"num_write_kb":508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":47,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"52'39","reported_seq":325,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:29.070333+0000","last_change":"2024-09-14T07:08:30.087672+0000","last_active":"2024-09-14T07:08:30.087672+0000","last_peered":"2024-09-14T07:08:29.070333+0000","last_clean":"2024-09-14T07:08:29.070333+0000","last_became_active":"2024-09-14T07:08:25.059209+0000","last_became_peered":"2024-09-14T07:08:25.059209+0000","last_unstale":"2024-09-14T07:08:29.070333+0000","last_undegraded":"2024-09-14T07:08:29.070333+0000","last_fullsized":"2024-09-14T07:08:29.070333+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":2156544,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":190,"num_read_kb":1526,"num_write":46,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":251904,"num_keys_recovered":6,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"52'255","reported_seq":1599,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.086985+0000","last_change":"2024-09-14T07:07:49.813689+0000","last_active":"2024-09-14T07:08:30.086985+0000","last_peered":"2024-09-14T07:08:30.086985+0000","last_clean":"2024-09-14T07:08:30.086985+0000","last_became_active":"2024-09-14T07:07:47.803551+0000","last_became_peered":"2024-09-14T07:07:47.803551+0000","last_unstale":"2024-09-14T07:08:30.086985+0000","last_undegraded":"2024-09-14T07:08:30.086985+0000","last_fullsized":"2024-09-14T07:08:30.086985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:04.303352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":29830364,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":2426,"num_read_kb":18834,"num_write":497,"num_write_kb":8782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"52'131","reported_seq":1879,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.095762+0000","last_change":"2024-09-14T07:07:49.820702+0000","last_active":"2024-09-14T07:08:30.095762+0000","last_peered":"2024-09-14T07:08:30.095762+0000","last_clean":"2024-09-14T07:08:30.095762+0000","last_became_active":"2024-09-14T07:07:47.781704+0000","last_became_peered":"2024-09-14T07:07:47.781704+0000","last_unstale":"2024-09-14T07:08:30.095762+0000","last_undegraded":"2024-09-14T07:08:30.095762+0000","last_fullsized":"2024-09-14T07:08:30.095762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:36:55.442304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":13044736,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":2904,"num_read_kb":7405,"num_write":82,"num_write_kb":514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"45'53","reported_seq":889,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.083728+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:08:30.083728+0000","last_peered":"2024-09-14T07:08:30.083728+0000","last_clean":"2024-09-14T07:08:30.083728+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:08:30.083728+0000","last_undegraded":"2024-09-14T07:08:30.083728+0000","last_fullsized":"2024-09-14T07:08:30.083728+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:35:28.959658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":15248384,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":753,"num_read_kb":4874,"num_write":39,"num_write_kb":972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"53'20","reported_seq":254,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:30.024353+0000","last_change":"2024-09-14T07:08:30.098915+0000","last_active":"2024-09-14T07:08:30.098915+0000","last_peered":"2024-09-14T07:08:30.024353+0000","last_clean":"2024-09-14T07:08:30.024353+0000","last_became_active":"2024-09-14T07:08:25.057927+0000","last_became_peered":"2024-09-14T07:08:25.057927+0000","last_unstale":"2024-09-14T07:08:30.024353+0000","last_undegraded":"2024-09-14T07:08:30.024353+0000","last_fullsized":"2024-09-14T07:08:30.024353+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":30841856,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":124,"num_read_kb":2016,"num_write":18,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":524288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"50'103","reported_seq":528,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.096171+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:08:30.096171+0000","last_peered":"2024-09-14T07:08:30.096171+0000","last_clean":"2024-09-14T07:08:30.096171+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:08:30.096171+0000","last_undegraded":"2024-09-14T07:08:30.096171+0000","last_fullsized":"2024-09-14T07:08:30.096171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:43:53.176138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":23587420,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":643,"num_read_kb":3022,"num_write":243,"num_write_kb":13316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"50'61","reported_seq":492,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.095169+0000","last_change":"2024-09-14T07:07:49.818154+0000","last_active":"2024-09-14T07:08:30.095169+0000","last_peered":"2024-09-14T07:08:30.095169+0000","last_clean":"2024-09-14T07:08:30.095169+0000","last_became_active":"2024-09-14T07:07:47.782423+0000","last_became_peered":"2024-09-14T07:07:47.782423+0000","last_unstale":"2024-09-14T07:08:30.095169+0000","last_undegraded":"2024-09-14T07:08:30.095169+0000","last_fullsized":"2024-09-14T07:08:30.095169+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:49:46.757482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":15085056,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":356,"num_read_kb":1179,"num_write":156,"num_write_kb":12283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"47'79","reported_seq":1113,"reported_epoch":53,"state":"peering","last_fresh":"2024-09-14T07:08:29.070161+0000","last_change":"2024-09-14T07:08:30.084772+0000","last_active":"2024-09-14T07:08:30.084772+0000","last_peered":"2024-09-14T07:08:29.070161+0000","last_clean":"2024-09-14T07:08:29.070161+0000","last_became_active":"2024-09-14T07:08:25.055416+0000","last_became_peered":"2024-09-14T07:08:25.055416+0000","last_unstale":"2024-09-14T07:08:29.070161+0000","last_undegraded":"2024-09-14T07:08:29.070161+0000","last_fullsized":"2024-09-14T07:08:29.070161+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":18952847,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":1173,"num_read_kb":2772,"num_write":67,"num_write_kb":1134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":664207,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"42'52","reported_seq":1329,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.083123+0000","last_change":"2024-09-14T07:08:19.007160+0000","last_active":"2024-09-14T07:08:30.083123+0000","last_peered":"2024-09-14T07:08:30.083123+0000","last_clean":"2024-09-14T07:08:30.083123+0000","last_became_active":"2024-09-14T07:08:19.006721+0000","last_became_peered":"2024-09-14T07:08:19.006721+0000","last_unstale":"2024-09-14T07:08:30.083123+0000","last_undegraded":"2024-09-14T07:08:30.083123+0000","last_fullsized":"2024-09-14T07:08:30.083123+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:51:33.424143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":34771456,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":1123,"num_read_kb":5969,"num_write":51,"num_write_kb":1028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"53'46","reported_seq":1380,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:08:30.085226+0000","last_change":"2024-09-14T07:07:49.804715+0000","last_active":"2024-09-14T07:08:30.085226+0000","last_peered":"2024-09-14T07:08:30.085226+0000","last_clean":"2024-09-14T07:08:30.085226+0000","last_became_active":"2024-09-14T07:07:47.787058+0000","last_became_peered":"2024-09-14T07:07:47.787058+0000","last_unstale":"2024-09-14T07:08:30.085226+0000","last_undegraded":"2024-09-14T07:08:30.085226+0000","last_fullsized":"2024-09-14T07:08:30.085226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:58.784918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":54760976,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":1265,"num_read_kb":11261,"num_write":53,"num_write_kb":2474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":699532956,"num_objects":351,"num_object_clones":50,"num_object_copies":702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":351,"num_whiteouts":120,"num_read":39454,"num_read_kb":198032,"num_write":3723,"num_write_kb":136348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":16104881,"num_keys_recovered":70,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16867328,"data_stored":244100002,"data_compressed":4559541,"data_compressed_allocated":14860288,"data_compressed_original":242892800,"omap_allocated":0,"internal_metadata":0},"log_size":2577,"ondisk_log_size":2577,"up":64,"acting":62,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12116,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37408,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334432,"statfs":{"total":96636764160,"available":96598458368,"internally_reserved":0,"allocated":2109440,"data_stored":17894541,"data_compressed":391176,"data_compressed_allocated":1294336,"data_compressed_original":17514496,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607568,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37276,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334564,"statfs":{"total":96636764160,"available":96598593536,"internally_reserved":0,"allocated":1974272,"data_stored":11455655,"data_compressed":525489,"data_compressed_allocated":1044480,"data_compressed_original":11024384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607567,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":4712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":4825088,"data_stored":60011962,"data_compressed":1041438,"data_compressed_allocated":3727360,"data_compressed_original":59443200,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607569,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38808,"kb_used_data":3460,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333032,"statfs":{"total":96636764160,"available":96597024768,"internally_reserved":0,"allocated":3543040,"data_stored":47238621,"data_compressed":695967,"data_compressed_allocated":2600960,"data_compressed_original":46815232,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,6],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607569,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38772,"kb_used_data":3416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333068,"statfs":{"total":96636764160,"available":96597061632,"internally_reserved":0,"allocated":3497984,"data_stored":49144168,"data_compressed":766464,"data_compressed_allocated":2641920,"data_compressed_original":48777216,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607569,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39180,"kb_used_data":3820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332660,"statfs":{"total":96636764160,"available":96596643840,"internally_reserved":0,"allocated":3911680,"data_stored":57029321,"data_compressed":793348,"data_compressed_allocated":3022848,"data_compressed_original":56627200,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":56201023,"data_compressed":787290,"data_compressed_allocated":2985984,"data_compressed_original":56053760,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2895872,"data_stored":48911483,"data_compressed":766464,"data_compressed_allocated":2641920,"data_compressed_original":48777216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2945024,"data_stored":47005956,"data_compressed":695967,"data_compressed_allocated":2600960,"data_compressed_original":46815232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4362240,"data_stored":60498112,"data_compressed":1129587,"data_compressed_allocated":3862528,"data_compressed_original":60168192,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1859584,"data_stored":20032106,"data_compressed":519217,"data_compressed_allocated":1576960,"data_compressed_original":19849216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1560576,"data_stored":11451322,"data_compressed":661016,"data_compressed_allocated":1191936,"data_compressed_original":11229184,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:08:35.981 INFO:tasks.thrashosds.thrasher:Setting 3.13 to [2, 0, 3, 1] 2024-09-14T07:08:35.981 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.13', '2', '0', '3', '1'] 2024-09-14T07:08:35.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.13 2 0 3 1 2024-09-14T07:08:35.997 INFO:teuthology.orchestra.run.smithi145.stdout:1143 write 0x6149600 thru 0x614a965 (0x1366 bytes) 2024-09-14T07:08:36.025 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:36.026 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:36.026 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:36.026 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:36.034 INFO:teuthology.orchestra.run.smithi145.stdout:1144 write 0xa1b2600 thru 0xa1c00e7 (0xdae8 bytes) 2024-09-14T07:08:36.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:36.054 INFO:teuthology.orchestra.run.smithi145.stdout:1148 trunc from 0xa1c0200 to 0x24b1a00 2024-09-14T07:08:36.055 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:36.053+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:36.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:36.096 INFO:teuthology.orchestra.run.smithi145.stdout:1149 read 0xc57800 thru 0xc5c084 (0x4885 bytes) 2024-09-14T07:08:36.148 INFO:teuthology.orchestra.run.smithi145.stdout:1152 write 0xd9faa00 thru 0xda0696c (0xbf6d bytes) 2024-09-14T07:08:36.149 INFO:teuthology.orchestra.run.smithi145.stdout:1153 write 0xb0ab400 thru 0xb0b532a (0x9f2b bytes) 2024-09-14T07:08:36.150 INFO:teuthology.orchestra.run.smithi145.stdout:1154 write 0xa0bde00 thru 0xa0ca369 (0xc56a bytes) 2024-09-14T07:08:36.150 INFO:teuthology.orchestra.run.smithi145.stdout:1159 read 0xc191a00 thru 0xc192d3d (0x133e bytes) 2024-09-14T07:08:36.150 INFO:teuthology.orchestra.run.smithi145.stdout:1160 read 0x67f4e00 thru 0x6802391 (0xd592 bytes) 2024-09-14T07:08:36.151 INFO:teuthology.orchestra.run.smithi145.stdout:1161 punch from 0xa924400 to 0xa92ae00, (0x6a00 bytes) 2024-09-14T07:08:36.159 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:36.153+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:36.188 INFO:teuthology.orchestra.run.smithi145.stdout:1162 write 0xe80dc00 thru 0xe81aa5f (0xce60 bytes) 2024-09-14T07:08:36.192 INFO:teuthology.orchestra.run.smithi145.stdout:1164 punch from 0xaf2c000 to 0xaf34000, (0x8000 bytes) 2024-09-14T07:08:36.209 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:36.209 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:36.209 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:36.209 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:36.209 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:36.217 INFO:teuthology.orchestra.run.smithi145.stdout:1166 trunc from 0xe81ac00 to 0x5120a00 2024-09-14T07:08:36.221 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:36.222 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:36.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:36.240 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:36.240 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:36.256 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:36.256 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:36.253+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:36.259 INFO:teuthology.orchestra.run.smithi145.stdout:1168 trunc from 0x5120a00 to 0xe9ae00 2024-09-14T07:08:36.277 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:36.295 INFO:teuthology.orchestra.run.smithi145.stdout:1169 trunc from 0xe9ae00 to 0x44a7200 2024-09-14T07:08:36.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:36.318 INFO:teuthology.orchestra.run.smithi145.stdout:1172 read 0x303c00 thru 0x304f16 (0x1317 bytes) 2024-09-14T07:08:36.320 INFO:teuthology.orchestra.run.smithi145.stdout:1173 write 0x654200 thru 0x65b728 (0x7529 bytes) 2024-09-14T07:08:36.323 INFO:teuthology.orchestra.run.smithi145.stdout:1174 write 0x2626000 thru 0x262749b (0x149c bytes) 2024-09-14T07:08:36.324 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:36.324 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:36.324 INFO:teuthology.orchestra.run.smithi145.stdout:1177 write 0x2215a00 thru 0x2222374 (0xc975 bytes) 2024-09-14T07:08:36.354 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:36.354 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:36.357 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:36.353+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:36.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:36.370 INFO:teuthology.orchestra.run.smithi145.stdout:1178 write 0x96b6800 thru 0x96c6161 (0xf962 bytes) 2024-09-14T07:08:36.371 INFO:teuthology.orchestra.run.smithi145.stdout:1179 read 0x64ef000 thru 0x64f013d (0x113e bytes) 2024-09-14T07:08:36.371 INFO:teuthology.orchestra.run.smithi145.stdout:1180 write 0x4070c00 thru 0x407ef41 (0xe342 bytes) 2024-09-14T07:08:36.372 INFO:teuthology.orchestra.run.smithi145.stdout:1183 trunc from 0x96c6200 to 0x97d0a00 2024-09-14T07:08:36.403 INFO:teuthology.orchestra.run.smithi145.stdout:1184 read 0xf4f600 thru 0xf5d079 (0xda7a bytes) 2024-09-14T07:08:36.407 INFO:teuthology.orchestra.run.smithi145.stdout:1185 read 0x55ed000 thru 0x55ee256 (0x1257 bytes) 2024-09-14T07:08:36.408 INFO:teuthology.orchestra.run.smithi145.stdout:1188 write 0x7f0f600 thru 0x7f19be5 (0xa5e6 bytes) 2024-09-14T07:08:36.409 INFO:teuthology.orchestra.run.smithi145.stdout:1191 write 0x6b07800 thru 0x6b0f0f2 (0x78f3 bytes) 2024-09-14T07:08:36.444 INFO:teuthology.orchestra.run.smithi145.stdout:1194 write 0xdfb2400 thru 0xdfb39a1 (0x15a2 bytes) 2024-09-14T07:08:36.458 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:36.453+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:36.459 INFO:teuthology.orchestra.run.smithi145.stdout:1196 trunc from 0xdfb3a00 to 0x8c3ea00 2024-09-14T07:08:36.498 INFO:teuthology.orchestra.run.smithi145.stdout:1197 read 0x6bc4400 thru 0x6bc60f4 (0x1cf5 bytes) 2024-09-14T07:08:36.514 INFO:teuthology.orchestra.run.smithi145.stdout:1201 trunc from 0x8c3ea00 to 0x39f7400 2024-09-14T07:08:36.528 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:36.529 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:36.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:36.575 INFO:teuthology.orchestra.run.smithi145.stdout:1203 write 0x4e93e00 thru 0x4e9e1ff (0xa400 bytes) 2024-09-14T07:08:36.587 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:36.585+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:36.612 INFO:teuthology.orchestra.run.smithi145.stdout:1207 write 0x7536c00 thru 0x7538bec (0x1fed bytes) 2024-09-14T07:08:36.630 INFO:teuthology.orchestra.run.smithi145.stdout:1208 trunc from 0x7538c00 to 0x3cd800 2024-09-14T07:08:36.659 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:36.659 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:36.660 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:36.657+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:36.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:36.676 INFO:teuthology.orchestra.run.smithi145.stdout:1210 punch from 0x35c800 to 0x366c00, (0xa400 bytes) 2024-09-14T07:08:36.678 INFO:teuthology.orchestra.run.smithi145.stdout:1213 read 0x67800 thru 0x72b49 (0xb34a bytes) 2024-09-14T07:08:36.715 INFO:teuthology.orchestra.run.smithi145.stdout:1215 write 0x503d600 thru 0x503e9e1 (0x13e2 bytes) 2024-09-14T07:08:36.732 INFO:teuthology.orchestra.run.smithi145.stdout:1219 trunc from 0x503ea00 to 0xa355200 2024-09-14T07:08:36.756 INFO:teuthology.orchestra.run.smithi145.stdout:1220 read 0x8912600 thru 0x892253c (0xff3d bytes) 2024-09-14T07:08:36.759 INFO:teuthology.orchestra.run.smithi145.stdout:1222 read 0x137c000 thru 0x13897cd (0xd7ce bytes) 2024-09-14T07:08:36.761 INFO:teuthology.orchestra.run.smithi145.stdout:1223 read 0x82a6e00 thru 0x82b6579 (0xf77a bytes) 2024-09-14T07:08:36.761 INFO:teuthology.orchestra.run.smithi145.stdout:1224 write 0x4068200 thru 0x4076ae3 (0xe8e4 bytes) 2024-09-14T07:08:36.763 INFO:teuthology.orchestra.run.smithi145.stdout:1226 punch from 0x6999000 to 0x699de00, (0x4e00 bytes) 2024-09-14T07:08:36.763 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:36.761+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:36.764 INFO:teuthology.orchestra.run.smithi145.stdout:1227 punch from 0x93bc00 to 0x943000, (0x7400 bytes) 2024-09-14T07:08:36.764 INFO:teuthology.orchestra.run.smithi145.stdout:1228 read 0x780a000 thru 0x7810395 (0x6396 bytes) 2024-09-14T07:08:36.782 INFO:teuthology.orchestra.run.smithi145.stdout:1229 trunc from 0xa355200 to 0x169600 2024-09-14T07:08:36.835 INFO:teuthology.orchestra.run.smithi145.stdout:1231 read 0x130e00 thru 0x13eadf (0xdce0 bytes) 2024-09-14T07:08:36.837 INFO:teuthology.orchestra.run.smithi145.stdout:1233 punch from 0x8800 to 0xec00, (0x6400 bytes) 2024-09-14T07:08:36.838 INFO:teuthology.orchestra.run.smithi145.stdout:1236 read 0x13f200 thru 0x14954e (0xa34f bytes) 2024-09-14T07:08:36.847 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:36.847 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:36.857 INFO:teuthology.orchestra.run.smithi145.stdout:1237 trunc from 0x169600 to 0x792a000 2024-09-14T07:08:36.861 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:36.862 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:36.857+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:36.892 INFO:teuthology.orchestra.run.smithi145.stdout:1238 trunc from 0x792a000 to 0xc4d3e00 2024-09-14T07:08:36.907 INFO:teuthology.orchestra.run.smithi145.stdout:1243 write 0x358da00 thru 0x3590bcb (0x31cc bytes) 2024-09-14T07:08:36.908 INFO:teuthology.orchestra.run.smithi145.stdout:1244 write 0xb55b400 thru 0xb56854e (0xd14f bytes) 2024-09-14T07:08:36.908 INFO:teuthology.orchestra.run.smithi145.stdout:1245 read 0x33cbe00 thru 0x33d3721 (0x7922 bytes) 2024-09-14T07:08:36.909 INFO:teuthology.orchestra.run.smithi145.stdout:1248 write 0xb006c00 thru 0xb00d1fe (0x65ff bytes) 2024-09-14T07:08:36.909 INFO:teuthology.orchestra.run.smithi145.stdout:1249 read 0x8819200 thru 0x882142a (0x822b bytes) 2024-09-14T07:08:36.909 INFO:teuthology.orchestra.run.smithi145.stdout:1250 read 0x24db200 thru 0x24e49ce (0x97cf bytes) 2024-09-14T07:08:36.911 INFO:teuthology.orchestra.run.smithi145.stdout:1253 read 0xc1e5c00 thru 0xc1e5d5c (0x15d bytes) 2024-09-14T07:08:36.912 INFO:teuthology.orchestra.run.smithi145.stdout:1254 write 0x73fac00 thru 0x74078f0 (0xccf1 bytes) 2024-09-14T07:08:36.912 INFO:teuthology.orchestra.run.smithi145.stdout:1257 read 0x573b800 thru 0x57442ec (0x8aed bytes) 2024-09-14T07:08:36.912 INFO:teuthology.orchestra.run.smithi145.stdout:1258 write 0x7a2dc00 thru 0x7a2dc19 (0x1a bytes) 2024-09-14T07:08:36.915 INFO:teuthology.orchestra.run.smithi145.stdout:1259 trunc from 0xc4d3e00 to 0xb415e00 2024-09-14T07:08:36.961 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:36.957+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:36.973 INFO:teuthology.orchestra.run.smithi145.stdout:1263 write 0xd217a00 thru 0xd21e57c (0x6b7d bytes) 2024-09-14T07:08:36.974 INFO:teuthology.orchestra.run.smithi145.stdout:1267 read 0xb29800 thru 0xb2d112 (0x3913 bytes) 2024-09-14T07:08:36.975 INFO:teuthology.orchestra.run.smithi145.stdout:1268 read 0x474f000 thru 0x475a5f2 (0xb5f3 bytes) 2024-09-14T07:08:36.977 INFO:teuthology.orchestra.run.smithi145.stdout:1270 write 0x9fc4600 thru 0x9fcf32f (0xad30 bytes) 2024-09-14T07:08:36.977 INFO:teuthology.orchestra.run.smithi145.stdout:1272 punch from 0x66b3800 to 0x66b6200, (0x2a00 bytes) 2024-09-14T07:08:36.978 INFO:teuthology.orchestra.run.smithi145.stdout:1275 write 0xc5af400 thru 0xc5bb75e (0xc35f bytes) 2024-09-14T07:08:36.978 INFO:teuthology.orchestra.run.smithi145.stdout:1276 read 0x545fe00 thru 0x54645e9 (0x47ea bytes) 2024-09-14T07:08:36.979 INFO:teuthology.orchestra.run.smithi145.stdout:1278 punch from 0x373200 to 0x37e000, (0xae00 bytes) 2024-09-14T07:08:36.980 INFO:teuthology.orchestra.run.smithi145.stdout:1281 punch from 0x7232a00 to 0x7240800, (0xde00 bytes) 2024-09-14T07:08:37.011 INFO:teuthology.orchestra.run.smithi145.stdout:1283 write 0xe9aba00 thru 0xe9b03b3 (0x49b4 bytes) 2024-09-14T07:08:37.015 INFO:teuthology.orchestra.run.smithi145.stdout:1284 punch from 0x3d68000 to 0x3d6c800, (0x4800 bytes) 2024-09-14T07:08:37.016 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:37.016 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:37.016 INFO:teuthology.orchestra.run.smithi145.stdout:1286 read 0xb40d600 thru 0xb41d052 (0xfa53 bytes) 2024-09-14T07:08:37.017 INFO:teuthology.orchestra.run.smithi145.stdout:1291 write 0xb14600 thru 0xb23e78 (0xf879 bytes) 2024-09-14T07:08:37.019 INFO:teuthology.orchestra.run.smithi145.stdout:1292 punch from 0x6150c00 to 0x6157c00, (0x7000 bytes) 2024-09-14T07:08:37.019 INFO:teuthology.orchestra.run.smithi145.stdout:1293 read 0x5542600 thru 0x555229f (0xfca0 bytes) 2024-09-14T07:08:37.021 INFO:teuthology.orchestra.run.smithi145.stdout:1295 punch from 0x4681200 to 0x468d200, (0xc000 bytes) 2024-09-14T07:08:37.021 INFO:teuthology.orchestra.run.smithi145.stdout:1297 read 0x243b600 thru 0x243fc6c (0x466d bytes) 2024-09-14T07:08:37.023 INFO:teuthology.orchestra.run.smithi145.stdout:1300 write 0x1aa7c00 thru 0x1ab3bd0 (0xbfd1 bytes) 2024-09-14T07:08:37.029 INFO:teuthology.orchestra.run.smithi145.stdout:1302 read 0x3779200 thru 0x377c20c (0x300d bytes) 2024-09-14T07:08:37.030 INFO:teuthology.orchestra.run.smithi145.stdout:1305 write 0x45c4c00 thru 0x45ced55 (0xa156 bytes) 2024-09-14T07:08:37.031 INFO:teuthology.orchestra.run.smithi145.stdout:1307 punch from 0x9a3b000 to 0x9a46200, (0xb200 bytes) 2024-09-14T07:08:37.032 INFO:teuthology.orchestra.run.smithi145.stdout:1308 read 0x2626c00 thru 0x2627b10 (0xf11 bytes) 2024-09-14T07:08:37.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:37.042 INFO:teuthology.orchestra.run.smithi145.stdout:1309 read 0xc770400 thru 0xc77efbb (0xebbc bytes) 2024-09-14T07:08:37.043 INFO:teuthology.orchestra.run.smithi145.stdout:1311 read 0xd4afa00 thru 0xd4b0caa (0x12ab bytes) 2024-09-14T07:08:37.044 INFO:teuthology.orchestra.run.smithi145.stdout:1312 read 0xa561800 thru 0xa565638 (0x3e39 bytes) 2024-09-14T07:08:37.045 INFO:teuthology.orchestra.run.smithi145.stdout:1313 write 0x8445a00 thru 0x8447f36 (0x2537 bytes) 2024-09-14T07:08:37.047 INFO:teuthology.orchestra.run.smithi145.stdout:1314 read 0xbd11400 thru 0xbd1555c (0x415d bytes) 2024-09-14T07:08:37.048 INFO:teuthology.orchestra.run.smithi145.stdout:1315 write 0x6328600 thru 0x63365bd (0xdfbe bytes) 2024-09-14T07:08:37.048 INFO:teuthology.orchestra.run.smithi145.stdout:1317 read 0x702b000 thru 0x702ebd5 (0x3bd6 bytes) 2024-09-14T07:08:37.050 INFO:teuthology.orchestra.run.smithi145.stdout:1318 punch from 0x9b1ee00 to 0x9b2e200, (0xf400 bytes) 2024-09-14T07:08:37.050 INFO:teuthology.orchestra.run.smithi145.stdout:1319 write 0x9b0400 thru 0x9bbd17 (0xb918 bytes) 2024-09-14T07:08:37.064 INFO:teuthology.orchestra.run.smithi145.stdout:1321 trunc from 0xe9b0400 to 0x6e65e00 2024-09-14T07:08:37.075 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:37.073+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:37.108 INFO:teuthology.orchestra.run.smithi145.stdout:1324 read 0xd12a00 thru 0xd200d8 (0xd6d9 bytes) 2024-09-14T07:08:37.112 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.13 pg_upmap_items mapping to [2->0,3->1] 2024-09-14T07:08:37.114 INFO:teuthology.orchestra.run.smithi145.stdout:1326 trunc from 0x6e65e00 to 0x51e6400 2024-09-14T07:08:37.162 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:37.157+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:37.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:08:37.200 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:37.200 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:37.214 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:37.263 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:37.261+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:37.336 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:37.336 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:37.350 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:37.392 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:37.389+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:37.464 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:37.461+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:37.516 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:37.516 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:37.531 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:37.565 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:37.561+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:37.654 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:37.654 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:37.666 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:37.665+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:37.668 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:37.766 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:37.761+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:37.785 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:37.785 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:37.799 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:37.867 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:37.865+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:37.918 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:37.918 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:37.934 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:37.980 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:37.977+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:38.051 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:38.051 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:38.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:38.069 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:38.065+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:38.114 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:08:38.155 INFO:teuthology.orchestra.run.smithi145.stdout:1327 punch from 0xebd000 to 0xec6e00, (0x9e00 bytes) 2024-09-14T07:08:38.157 INFO:teuthology.orchestra.run.smithi145.stdout:1331 write 0x6c8800 thru 0x6cc0cb (0x38cc bytes) 2024-09-14T07:08:38.160 INFO:teuthology.orchestra.run.smithi145.stdout:1332 punch from 0x4c6fe00 to 0x4c79c00, (0x9e00 bytes) 2024-09-14T07:08:38.170 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:38.165+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:38.187 INFO:teuthology.orchestra.run.smithi145.stdout:1333 write 0xa93ea00 thru 0xa94321c (0x481d bytes) 2024-09-14T07:08:38.189 INFO:teuthology.orchestra.run.smithi145.stdout:1334 read 0x9d73e00 thru 0x9d756d6 (0x18d7 bytes) 2024-09-14T07:08:38.189 INFO:teuthology.orchestra.run.smithi145.stdout:1335 punch from 0x5105800 to 0x510ba00, (0x6200 bytes) 2024-09-14T07:08:38.189 INFO:teuthology.orchestra.run.smithi145.stdout:1336 punch from 0x7ecda00 to 0x7ed8000, (0xa600 bytes) 2024-09-14T07:08:38.197 INFO:teuthology.orchestra.run.smithi145.stdout:1337 trunc from 0xa943400 to 0xdf9b200 2024-09-14T07:08:38.216 INFO:teuthology.orchestra.run.smithi145.stdout:1339 read 0x3a6ec00 thru 0x3a7e2f6 (0xf6f7 bytes) 2024-09-14T07:08:38.218 INFO:teuthology.orchestra.run.smithi145.stdout:1340 read 0x657b400 thru 0x657cf90 (0x1b91 bytes) 2024-09-14T07:08:38.218 INFO:teuthology.orchestra.run.smithi145.stdout:1343 read 0x1cbfe00 thru 0x1ccb18f (0xb390 bytes) 2024-09-14T07:08:38.219 INFO:teuthology.orchestra.run.smithi145.stdout:1344 clone 11 order 24 su 4194304 sc 10 2024-09-14T07:08:38.245 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:38.245 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:38.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:38.307 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:38.305+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:38.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:38.387 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:38.385+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:38.394 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:38.394 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:38.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:38.473 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:38.469+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:38.483 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:38.500 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:38.500 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:38.517 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:38.517 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:38.538 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:38.559 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:38.559 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:38.574 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:38.574 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:38.574 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:38.569+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:38.581 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:38.581 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:38.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:38.675 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:38.673+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:38.767 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:368 3.1f 3:f8928d1a:::rbd_data.1161a9b4f57.0000000000000036:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:38.767 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.702131462000001, 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01725641, 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 368 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064164+0000", 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064165+0000", 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.540000000000001e-07 2024-09-14T07:08:38.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064170+0000", 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8940000000000001e-06 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064185+0000", 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5182e-05 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064193+0000", 2024-09-14T07:08:38.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7999999999999999e-06 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064281+0000", 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7517999999999998e-05 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064365+0000", 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4728000000000006e-05 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064567+0000", 2024-09-14T07:08:38.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201757 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068739+0000", 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041723519999999998 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081335+0000", 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012595492999999999 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081366+0000", 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1538999999999997e-05 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081421+0000", 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4193000000000001e-05 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:373 3.1f 3:fcd26671:::rbd_data.1161a9b4f57.0000000000000045:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:38.773 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.701778723, 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017078697, 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 373 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064517+0000", 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064518+0000", 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064524+0000", 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9839999999999998e-06 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064536+0000", 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2089999999999999e-05 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.064544+0000", 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1589999999999993e-06 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065143+0000", 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059867500000000001 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065216+0000", 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3497000000000005e-05 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.065416+0000", 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019953899999999999 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068744+0000", 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033285459999999999 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081472+0000", 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012727815 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081528+0000", 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5958000000000002e-05 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081596+0000", 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7489000000000004e-05 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4525.0:386 3.1f 3:fbac3711:::rbd_data.1161a9b4f57.000000000000005d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.70003488, 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015471182999999999, 2024-09-14T07:08:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4525", 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2415073884", 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 386 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066261+0000", 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066262+0000", 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5600000000000004e-07 2024-09-14T07:08:38.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066268+0000", 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0809999999999999e-06 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066279+0000", 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1389999999999999e-05 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066286+0000", 2024-09-14T07:08:38.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8959999999999997e-06 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066393+0000", 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107413 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066451+0000", 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7454000000000003e-05 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.066556+0000", 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105326 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.068748+0000", 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002192285 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081641+0000", 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012892280000000001 2024-09-14T07:08:38.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081688+0000", 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:07.081732+0000", 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4132999999999999e-05 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:286 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 409600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.298059540000001, 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01542496, 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 286 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468236+0000", 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468238+0000", 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3960000000000001e-06 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468244+0000", 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468267+0000", 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3603000000000001e-05 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468278+0000", 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0883000000000001e-05 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483169+0000", 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014890970999999999 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483418+0000", 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024896800000000002 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483661+0000", 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024296499999999999 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.789 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:287 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.298002568000001, 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015987674, 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 287 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468293+0000", 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468294+0000", 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6300000000000004e-07 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468329+0000", 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4799999999999999e-05 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468349+0000", 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0299999999999999e-05 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468363+0000", 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4284000000000001e-05 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.483821+0000", 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015458120000000001 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484068+0000", 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000246281 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484281+0000", 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000213026 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:288 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.297907867999999, 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016408030000000001, 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 288 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468388+0000", 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468389+0000", 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.64e-06 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468396+0000", 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468411+0000", 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5153e-05 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468419+0000", 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7039999999999994e-06 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484366+0000", 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015946689999999999 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484617+0000", 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025165400000000003 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484796+0000", 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017846299999999999 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:289 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 122880~6656] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.297863888999998, 2024-09-14T07:08:38.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111327999999999, 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 289 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468432+0000", 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468433+0000", 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1999999999999999e-07 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468436+0000", 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3969999999999998e-06 2024-09-14T07:08:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468445+0000", 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9800000000000004e-06 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468455+0000", 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6900000000000004e-06 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.484948+0000", 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016493799 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485216+0000", 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026811499999999998 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485543+0000", 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032662700000000002 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:290 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.297830842, 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017722122999999999, 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 290 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468465+0000", 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4000000000000001e-07 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468469+0000", 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7529999999999999e-06 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468478+0000", 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6799999999999999e-06 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468485+0000", 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.153e-06 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485735+0000", 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017249970999999999 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.485956+0000", 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022074799999999999 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486187+0000", 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023117799999999999 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:291 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 352256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.807 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.297799766000001, 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018193804000000001, 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 291 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:38.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468496+0000", 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468497+0000", 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468502+0000", 2024-09-14T07:08:38.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5439999999999998e-06 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468531+0000", 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8898999999999999e-05 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468543+0000", 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1756999999999999e-05 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486295+0000", 2024-09-14T07:08:38.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017751423999999998 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486492+0000", 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019711899999999999 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486690+0000", 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019808099999999999 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:292 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 417792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.297692962999999, 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018553717000000001, 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 292 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468603+0000", 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468604+0000", 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5999999999999999e-06 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468611+0000", 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7800000000000003e-06 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468633+0000", 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1710000000000001e-05 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468647+0000", 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4209999999999999e-05 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486764+0000", 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018116538000000001 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.486948+0000", 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018412999999999999 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487156+0000", 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020874899999999999 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:293 3.6 3:67336471:::rbd_data.1161a9b4f57.0000000000000086:7 [read 516096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.297626400999999, 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019096847, 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 293 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468669+0000", 2024-09-14T07:08:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468671+0000", 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468676+0000", 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.857e-06 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468694+0000", 2024-09-14T07:08:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7629999999999999e-05 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.468712+0000", 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7762999999999999e-05 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487288+0000", 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018576294 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487555+0000", 2024-09-14T07:08:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026692399999999998 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.487766+0000", 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021110899999999999 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.477554721000001, 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:08:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:08:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.448047089999999, 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:08:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:08:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.439399731000002, 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:08:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:08:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:08:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:341 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:b [sparse-read 9764864~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.150989673000002, 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015588305, 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 341 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615306+0000", 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615307+0000", 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615312+0000", 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615331+0000", 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8542e-05 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.615346+0000", 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4826e-05 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.629124+0000", 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013778367999999999 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.629369+0000", 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000244842 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.630894+0000", 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015252600000000001 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.773307390999999, 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:08:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.606578164999998, 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:08:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 16.424646651, 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:08:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.673577157, 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:08:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:08:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:08:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4732.0:183 3.1b 3:d96d4dd6:::rbd_data.1161bf39a192.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 6709248~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8237071169999997, 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030801473999999999, 2024-09-14T07:08:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4732", 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/303097868", 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 183 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942590+0000", 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944540+0000", 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001949824 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944564+0000", 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4199e-05 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944574+0000", 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944644+0000", 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0240999999999994e-05 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944798+0000", 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153432 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.945082+0000", 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028403899999999998 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.959401+0000", 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014319159999999999 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973083+0000", 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013682016 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973261+0000", 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017752999999999999 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973390+0000", 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012949500000000001 2024-09-14T07:08:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:38.863 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:38.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:38.865 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:38.861+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:38.965 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:38.965+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:38.994 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:38.994 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:38.994 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:39.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:39.066 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:39.065+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:39.135 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:39.135 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:39.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:39.167 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:39.161+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:39.267 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:39.265+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:39.281 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone10 from 0xdf9b200 (overlap 0x169600) to 0x0 2024-09-14T07:08:39.282 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:39.282 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:295 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.812050788000001, 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028784614, 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 295 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.283 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469603+0000", 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469619+0000", 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6163000000000001e-05 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469646+0000", 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6040000000000001e-05 2024-09-14T07:08:39.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469660+0000", 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4843e-05 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488319+0000", 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018658726 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488567+0000", 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024742500000000001 2024-09-14T07:08:39.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.489393+0000", 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082641099999999999 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493423+0000", 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040299050000000003 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498065+0000", 2024-09-14T07:08:39.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046422310000000001 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498254+0000", 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000189278 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498385+0000", 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013043899999999999 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1514 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065856110999999, 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028657183999999999, 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.289 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1514 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2300000000000002e-07 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215804+0000", 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.602e-06 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215811+0000", 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8190000000000003e-06 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215816+0000", 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2229999999999998e-06 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244228+0000", 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028411750999999999 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244350+0000", 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012267199999999999 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244452+0000", 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010169400000000001 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1515 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.293 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065828007, 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843423999999999, 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1515 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:39.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215824+0000", 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-09-14T07:08:39.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215826+0000", 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-06 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215832+0000", 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.947e-06 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215837+0000", 2024-09-14T07:08:39.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6570000000000002e-06 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244479+0000", 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028642223000000001 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244559+0000", 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9443000000000005e-05 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244666+0000", 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107538 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1517 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10027008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065807475, 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029071107999999998, 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.298 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1517 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215844+0000", 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215846+0000", 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.821e-06 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215852+0000", 2024-09-14T07:08:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215857+0000", 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6099999999999997e-06 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244719+0000", 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028862291000000002 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244803+0000", 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3523999999999995e-05 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244915+0000", 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111612 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1518 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065786797000001, 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029262857, 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1518 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215865+0000", 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5699999999999998e-07 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215867+0000", 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215878+0000", 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1399e-05 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215883+0000", 2024-09-14T07:08:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244948+0000", 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029064834000000001 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245029+0000", 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0693999999999997e-05 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245127+0000", 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8218000000000001e-05 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1520 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065759841999999, 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029524788999999999, 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1520 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215892+0000", 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:08:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215894+0000", 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6860000000000002e-06 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215902+0000", 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7929999999999998e-06 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215907+0000", 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245181+0000", 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029273648999999999 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245287+0000", 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010595299999999999 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245416+0000", 2024-09-14T07:08:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012906300000000001 2024-09-14T07:08:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1522 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065734331000002, 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029733393, 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1522 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9599999999999999e-07 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215919+0000", 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2749999999999998e-06 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215931+0000", 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215937+0000", 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245451+0000", 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029513475000000001 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245537+0000", 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5975000000000003e-05 2024-09-14T07:08:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245650+0000", 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011319099999999999 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1523 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13959168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065700922000001, 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029960937999999999, 2024-09-14T07:08:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1523 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215954+0000", 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3689999999999998e-06 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215961+0000", 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2420000000000003e-06 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215967+0000", 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8329999999999999e-06 2024-09-14T07:08:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245715+0000", 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029748177000000001 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245798+0000", 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3338999999999996e-05 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245911+0000", 2024-09-14T07:08:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112507 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1524 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065675527, 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030280225000000001, 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1524 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215976+0000", 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215978+0000", 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0509999999999999e-06 2024-09-14T07:08:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215985+0000", 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8120000000000002e-06 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215990+0000", 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5729999999999998e-06 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246034+0000", 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030043767999999998 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246121+0000", 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7093000000000001e-05 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246256+0000", 2024-09-14T07:08:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013446899999999999 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1525 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.065618953000001, 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030480007, 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1525 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216033+0000", 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0800000000000004e-07 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216036+0000", 2024-09-14T07:08:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8600000000000003e-06 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216045+0000", 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3720000000000005e-06 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216052+0000", 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246311+0000", 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030259496 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246397+0000", 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5339000000000004e-05 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246512+0000", 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000115393 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1664 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10092544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.060301200000001, 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028510918999999999, 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1664 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221350+0000", 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221354+0000", 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2339999999999999e-06 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221364+0000", 2024-09-14T07:08:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0445000000000001e-05 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221373+0000", 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4950000000000008e-06 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249615+0000", 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028241973 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249750+0000", 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013522499999999999 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249861+0000", 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110908 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1665 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.060267134, 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028695215, 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1665 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9699999999999996e-07 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221387+0000", 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221395+0000", 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0399999999999993e-06 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221401+0000", 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.806e-06 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249893+0000", 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028491951000000001 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249971+0000", 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8419999999999998e-05 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250079+0000", 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010798600000000001 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1667 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.060241279, 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028871159, 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1667 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221413+0000", 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.396e-06 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221422+0000", 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2250000000000006e-06 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221428+0000", 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0839999999999997e-06 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250112+0000", 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028684345 2024-09-14T07:08:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250191+0000", 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8571000000000004e-05 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250281+0000", 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0094999999999997e-05 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1668 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.060201860999999, 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02902478, 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1668 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221450+0000", 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1399999999999997e-07 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221453+0000", 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221462+0000", 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.884e-06 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221468+0000", 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5390000000000004e-06 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250311+0000", 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843160999999999 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250389+0000", 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7550000000000001e-05 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250474+0000", 2024-09-14T07:08:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4982999999999996e-05 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1669 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.060148566999999, 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029194801999999999, 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1669 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221503+0000", 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221506+0000", 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221516+0000", 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9450000000000005e-06 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221523+0000", 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:08:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250506+0000", 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028983992 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250609+0000", 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010213399999999999 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250697+0000", 2024-09-14T07:08:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8551e-05 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1670 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.060118296999999, 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029401132999999999, 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1670 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221536+0000", 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5550000000000001e-06 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221543+0000", 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4939999999999997e-06 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250744+0000", 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029195045999999999 2024-09-14T07:08:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250836+0000", 2024-09-14T07:08:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1589999999999996e-05 2024-09-14T07:08:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250934+0000", 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7826999999999997e-05 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1904 3.b 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9175040~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.99275832, 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.101088252, 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1904 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288894+0000", 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.666e-06 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315149+0000", 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026254493 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315177+0000", 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315200+0000", 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2949e-05 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315307+0000", 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107222 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315483+0000", 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017589 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.317437+0000", 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019539890000000002 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.384167+0000", 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066730085999999994 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389856+0000", 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056887079999999998 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389925+0000", 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8977e-05 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389981+0000", 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6158e-05 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1907 3.1a e43/21 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head v 43'107, mlcod=42'105)", 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.959473031999998, 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.068409926999999995, 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8299999999999998e-07 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331614+0000", 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094355310000000005 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331636+0000", 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1804e-05 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331668+0000", 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2094999999999997e-05 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331759+0000", 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0901000000000003e-05 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331787+0000", 2024-09-14T07:08:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8750000000000001e-05 2024-09-14T07:08:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390536+0000", 2024-09-14T07:08:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058748732999999997 2024-09-14T07:08:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390588+0000", 2024-09-14T07:08:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1829999999999997e-05 2024-09-14T07:08:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1911 3.8 e43/25 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head v 43'64, mlcod=42'61)", 2024-09-14T07:08:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.932251803, 2024-09-14T07:08:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.060945845999999998, 2024-09-14T07:08:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349401+0000", 2024-09-14T07:08:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.846e-06 2024-09-14T07:08:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354404+0000", 2024-09-14T07:08:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005003146 2024-09-14T07:08:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354427+0000", 2024-09-14T07:08:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2617999999999999e-05 2024-09-14T07:08:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354440+0000", 2024-09-14T07:08:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3559e-05 2024-09-14T07:08:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354525+0000", 2024-09-14T07:08:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4986000000000003e-05 2024-09-14T07:08:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354551+0000", 2024-09-14T07:08:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5276999999999999e-05 2024-09-14T07:08:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410278+0000", 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055727499999999999 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410345+0000", 2024-09-14T07:08:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6914000000000003e-05 2024-09-14T07:08:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:08:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.601865952000001, 2024-09-14T07:08:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:08:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:08:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:08:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:08:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:08:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:08:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.391 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.391 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:39.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:39.394 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:39.389+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:39.493 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:39.489+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:39.572 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:39.573 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:39.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:39.635 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:39.633+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:39.696 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:39.693+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:39.757 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:39.757 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:39.757 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:39.771 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T07:08:39.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:39.823 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:39.821+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:39.898 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:39.893+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:39.906 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1538 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684441381999999, 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054129517000000002, 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1538 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220949+0000", 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-06 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220959+0000", 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0932e-05 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220965+0000", 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.49e-06 2024-09-14T07:08:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274976+0000", 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054011221999999998 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275019+0000", 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2889000000000002e-05 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275076+0000", 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6455000000000001e-05 2024-09-14T07:08:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1539 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684414665999999, 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054229843, 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1539 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220976+0000", 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-06 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1069999999999999e-06 2024-09-14T07:08:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220988+0000", 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0119999999999996e-06 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275106+0000", 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054118053999999999 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275149+0000", 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2830000000000002e-05 2024-09-14T07:08:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275203+0000", 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4044e-05 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1540 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.68439235, 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05433267, 2024-09-14T07:08:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1540 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220996+0000", 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220998+0000", 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9759999999999998e-06 2024-09-14T07:08:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221009+0000", 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.242e-06 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275233+0000", 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054223461000000001 2024-09-14T07:08:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275275+0000", 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2726999999999999e-05 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275328+0000", 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2332e-05 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1541 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684370154, 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054437230000000003, 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1541 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:39.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221018+0000", 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2e-07 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221020+0000", 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3530000000000002e-06 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221026+0000", 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3110000000000002e-06 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221031+0000", 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275358+0000", 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054326722000000001 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275401+0000", 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2849000000000003e-05 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275454+0000", 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3844000000000002e-05 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1542 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:39.924 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684349740000002, 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054541385999999997, 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1542 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3700000000000001e-07 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221040+0000", 2024-09-14T07:08:39.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8059999999999999e-06 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221046+0000", 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1009999999999997e-06 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221051+0000", 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6940000000000003e-06 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275484+0000", 2024-09-14T07:08:39.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054433003000000001 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275527+0000", 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3031999999999997e-05 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275579+0000", 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2413000000000001e-05 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1543 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684329984000001, 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054648398000000001, 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1543 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221058+0000", 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4299999999999999e-07 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221060+0000", 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7990000000000001e-06 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221066+0000", 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221070+0000", 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7010000000000004e-06 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275609+0000", 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054538968 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275652+0000", 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2864000000000002e-05 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275706+0000", 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3597000000000002e-05 2024-09-14T07:08:39.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1545 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684310318000001, 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054855590000000003, 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1545 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4499999999999998e-07 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221085+0000", 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0330000000000008e-06 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221092+0000", 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.793e-06 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221098+0000", 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3090000000000003e-06 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275744+0000", 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054646404000000003 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275821+0000", 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6793999999999997e-05 2024-09-14T07:08:39.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275933+0000", 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111912 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1583 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 524288~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684282154999998, 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055023028000000002, 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1583 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221106+0000", 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221108+0000", 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0229999999999999e-06 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221121+0000", 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3645000000000001e-05 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221127+0000", 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.541e-06 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275735+0000", 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054608372000000002 2024-09-14T07:08:39.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275817+0000", 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2083999999999995e-05 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276128+0000", 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031090000000000002 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.941 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1585 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684252320999999, 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055153874999999998, 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1585 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221136+0000", 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7300000000000001e-07 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221138+0000", 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221144+0000", 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5100000000000004e-06 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221149+0000", 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.843e-06 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276165+0000", 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055015677999999998 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276231+0000", 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5959999999999999e-05 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276289+0000", 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8100000000000003e-05 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1586 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684228706999999, 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055262004000000003, 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1586 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221166+0000", 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3180000000000001e-06 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221173+0000", 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221178+0000", 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2050000000000001e-06 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276321+0000", 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055142663000000001 2024-09-14T07:08:39.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276364+0000", 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3615000000000002e-05 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276421+0000", 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6453000000000003e-05 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.950 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1588 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684201470000001, 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055364653999999999, 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1588 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221189+0000", 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3269999999999999e-06 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221195+0000", 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2169999999999999e-06 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221200+0000", 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:08:39.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276451+0000", 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055251452999999999 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276497+0000", 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5114000000000002e-05 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276551+0000", 2024-09-14T07:08:39.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4079999999999997e-05 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1589 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684180806000001, 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055473737000000002, 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.955 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1589 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221209+0000", 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9269999999999999e-06 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221215+0000", 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3069999999999999e-06 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221220+0000", 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6369999999999998e-06 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276582+0000", 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055361856000000001 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276625+0000", 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2803000000000003e-05 2024-09-14T07:08:39.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276680+0000", 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5789999999999999e-05 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1591 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:39.959 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.684160446, 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055584770999999998, 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1591 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:39.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222763+0000", 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001535914 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222789+0000", 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6194e-05 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222862+0000", 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2529999999999998e-05 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222953+0000", 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1185000000000002e-05 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276713+0000", 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053760371000000001 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276758+0000", 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4932000000000003e-05 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276812+0000", 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3644999999999999e-05 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1593 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 6225920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.682389771, 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054086803000000003, 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1593 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:39.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223000+0000", 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.289e-06 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223007+0000", 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.058e-06 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223013+0000", 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5779999999999998e-06 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276973+0000", 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053960263000000001 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277028+0000", 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000002e-05 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277084+0000", 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6357999999999998e-05 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1594 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.682365906000001, 2024-09-14T07:08:39.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054193419, 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1594 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223035+0000", 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2994e-05 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223043+0000", 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6539999999999999e-06 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223048+0000", 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6149999999999996e-06 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277114+0000", 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054066256 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277157+0000", 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2629000000000001e-05 2024-09-14T07:08:39.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277215+0000", 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7803999999999997e-05 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1598 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.681746351000001, 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054092236000000002, 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1598 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224174+0000", 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000533265 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224179+0000", 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2420000000000002e-06 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224190+0000", 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-05 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224197+0000", 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3549999999999999e-06 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277635+0000", 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053437314999999999 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277679+0000", 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4628e-05 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277733+0000", 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4131000000000001e-05 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:39.976 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.605080715, 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:08:39.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:08:39.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:08:39.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:08:39.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.560090409000001, 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.983 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:08:39.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.258157953000001, 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:08:39.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:39.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:39.994 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.936830219000001, 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:08:39.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:39.999 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:40.000 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:40.000 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:40.001 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:40.002 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:39.997+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:40.102 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:40.097+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:40.133 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:40.133 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:40.147 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:08:40.149 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:40.203 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:40.201+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:40.304 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:40.301+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:40.321 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:40.322 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:40.322 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:40.322 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:40.322 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:40.336 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:40.423 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:40.421+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:40.506 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:40.501+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:40.509 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.206740503999995, 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:40.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:40.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:40.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:40.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:40.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:40.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1502 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.286376143999998, 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042080580999999999, 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1502 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220869+0000", 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2300000000000001e-06 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220888+0000", 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8607e-05 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220907+0000", 2024-09-14T07:08:40.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9422000000000001e-05 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220922+0000", 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4375999999999999e-05 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262777+0000", 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855537999999998 2024-09-14T07:08:40.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262833+0000", 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5729000000000001e-05 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262949+0000", 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115679 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1503 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.286301332000001, 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042296750000000001, 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1503 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220945+0000", 2024-09-14T07:08:40.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.013e-06 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220951+0000", 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.806e-06 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220970+0000", 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8649999999999999e-05 2024-09-14T07:08:40.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3964999999999999e-05 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262989+0000", 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042005845 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263116+0000", 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012688 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263240+0000", 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012359099999999999 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1504 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:40.526 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.286242208000001, 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042493942999999999, 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1504 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:40.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.063e-06 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221011+0000", 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8059999999999997e-06 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221030+0000", 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8652999999999999e-05 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221044+0000", 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3879999999999999e-05 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263275+0000", 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042231869999999998 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263365+0000", 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9813000000000002e-05 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263496+0000", 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000130858 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1634 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.283048145999999, 2024-09-14T07:08:40.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041961863000000002, 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1634 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224198+0000", 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.77e-06 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224205+0000", 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6429999999999998e-06 2024-09-14T07:08:40.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224240+0000", 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5714999999999999e-05 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224555+0000", 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031487300000000002 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265911+0000", 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041355780000000002 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266040+0000", 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012856099999999999 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266158+0000", 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118521 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.535 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1636 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.282514521, 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042143574000000003, 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1636 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224731+0000", 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5600000000000001e-06 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224738+0000", 2024-09-14T07:08:40.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224756+0000", 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8153999999999999e-05 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224770+0000", 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4168999999999999e-05 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266618+0000", 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041847662000000001 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266730+0000", 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000111993 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266873+0000", 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014374400000000001 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1643 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 8978432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.281433486000001, 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042103224000000002, 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1643 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225812+0000", 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.207e-06 2024-09-14T07:08:40.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225818+0000", 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225836+0000", 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8221999999999999e-05 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225849+0000", 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.358e-05 2024-09-14T07:08:40.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267705+0000", 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855723999999997 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267805+0000", 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9437000000000004e-05 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267914+0000", 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109554 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.543 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1644 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.281375258000001, 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042307838, 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1644 2024-09-14T07:08:40.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225870+0000", 2024-09-14T07:08:40.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2270000000000001e-06 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225876+0000", 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2569999999999998e-06 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225894+0000", 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8385000000000001e-05 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225936+0000", 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2159000000000001e-05 2024-09-14T07:08:40.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267952+0000", 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04201585 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268067+0000", 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114985 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268177+0000", 2024-09-14T07:08:40.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109975 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1645 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.281286881, 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042484558999999998, 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.548 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1645 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:40.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225959+0000", 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225978+0000", 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9466e-05 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225998+0000", 2024-09-14T07:08:40.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9315999999999999e-05 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226013+0000", 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268213+0000", 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200410000000001 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268324+0000", 2024-09-14T07:08:40.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011123799999999999 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268442+0000", 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117889 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1646 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:40.552 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.281210503000001, 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042595283999999997, 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1646 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:40.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226035+0000", 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5060000000000001e-06 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226042+0000", 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4370000000000004e-06 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226060+0000", 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8266000000000001e-05 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226074+0000", 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4029000000000001e-05 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042405023999999999 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268551+0000", 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1973999999999999e-05 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268629+0000", 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8047999999999995e-05 2024-09-14T07:08:40.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1647 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12910592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.281152468999998, 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042800796000000002, 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.557 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1647 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226093+0000", 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.46e-06 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226099+0000", 2024-09-14T07:08:40.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3530000000000002e-06 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226118+0000", 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9055999999999998e-05 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226132+0000", 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268673+0000", 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042541272999999998 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268775+0000", 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102302 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268893+0000", 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117539 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.561 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1648 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.281093328000001, 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042904468000000001, 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1648 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226152+0000", 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.234e-06 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226157+0000", 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2000000000000002e-06 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226175+0000", 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7059e-05 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226204+0000", 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9536e-05 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268938+0000", 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042733698000000001 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268992+0000", 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4693e-05 2024-09-14T07:08:40.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269056+0000", 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3047999999999997e-05 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1649 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.28101912, 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043088794999999999, 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1649 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226227+0000", 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.33e-06 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226232+0000", 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.733e-06 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226251+0000", 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8816000000000001e-05 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226264+0000", 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-05 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269097+0000", 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0428324 2024-09-14T07:08:40.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269197+0000", 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100419 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269314+0000", 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116897 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1651 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.570 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.280960272000002, 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043199128000000003, 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1651 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226285+0000", 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.237e-06 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226291+0000", 2024-09-14T07:08:40.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1789999999999999e-06 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226311+0000", 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0656999999999999e-05 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226327+0000", 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5948999999999999e-05 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269352+0000", 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043024830999999999 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269419+0000", 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7281999999999997e-05 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269483+0000", 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3992999999999995e-05 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1700 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 32768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.280896943999998, 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043417208999999998, 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1700 2024-09-14T07:08:40.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226349+0000", 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3629999999999999e-06 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226354+0000", 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226372+0000", 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7612999999999999e-05 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226386+0000", 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4146e-05 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269525+0000", 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043139380999999997 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269631+0000", 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105574 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269765+0000", 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133759 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1702 3.13 3:cfeabdd5:::rbd_data.11616dafc67f.000000000000000f:9 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.280808524000001, 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043560261000000003, 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.579 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1702 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226437+0000", 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.567e-06 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226457+0000", 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395999999999999e-05 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226477+0000", 2024-09-14T07:08:40.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9998999999999999e-05 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226491+0000", 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4503e-05 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269835+0000", 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043343995000000003 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269914+0000", 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9018999999999997e-05 2024-09-14T07:08:40.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269996+0000", 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1781999999999997e-05 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1740 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.280731629999998, 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043673828999999997, 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.583 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1740 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226514+0000", 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4869999999999999e-06 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226520+0000", 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.976e-06 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226538+0000", 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7972999999999999e-05 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226551+0000", 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3142999999999999e-05 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270041+0000", 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043490174999999999 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270101+0000", 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9515999999999997e-05 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270187+0000", 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5558999999999999e-05 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:40.587 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.215957369000002, 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:08:40.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:08:40.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.592 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4803.0:1903 3.1f e43/21 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head v 43'21, mlcod=42'19)", 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.198796550000001, 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.057064131999999997, 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308449+0000", 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.279e-06 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324156+0000", 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015707199000000002 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324195+0000", 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9070999999999999e-05 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324217+0000", 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1752000000000001e-05 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324333+0000", 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011607000000000001 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324359+0000", 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6296e-05 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365477+0000", 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041117065000000001 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365512+0000", 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-05 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4703.0:530 3.f e46/21 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head v 46'75, mlcod=45'73)", 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.644857604999999, 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.075870370000000006, 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:40.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862389+0000", 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6579999999999998e-06 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870883+0000", 2024-09-14T07:08:40.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084939100000000003 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870916+0000", 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2428000000000002e-05 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870931+0000", 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5656000000000001e-05 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:40.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871022+0000", 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1063000000000005e-05 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871047+0000", 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4125e-05 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938212+0000", 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067164920000000003 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:40.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938257+0000", 2024-09-14T07:08:40.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5609999999999999e-05 2024-09-14T07:08:40.601 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.601 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.601 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.601 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:40.601 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:40.601 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:40.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:40.602 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:40.643 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:40.641+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:40.707 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:40.705+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:40.757 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:40.772 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:40.772 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:40.772 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:40.772 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:40.773 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:40.786 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:40.791 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:40.808 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:40.805+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:40.809 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:40.828 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:40.828 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:40.845 INFO:teuthology.orchestra.run.smithi145.stdout:1345 punch from 0xd36f800 to 0xd372a00, (0x3200 bytes) 2024-09-14T07:08:40.846 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:40.846 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:40.858 INFO:teuthology.orchestra.run.smithi145.stdout:1346 read 0x392b000 thru 0x393115e (0x615f bytes) 2024-09-14T07:08:40.860 INFO:teuthology.orchestra.run.smithi145.stdout:1347 read 0x6dafa00 thru 0x6db9065 (0x9666 bytes) 2024-09-14T07:08:40.861 INFO:teuthology.orchestra.run.smithi145.stdout:1349 write 0x3ab6a00 thru 0x3ac601c (0xf61d bytes) 2024-09-14T07:08:40.862 INFO:teuthology.orchestra.run.smithi145.stdout:1354 punch from 0x4089600 to 0x4092800, (0x9200 bytes) 2024-09-14T07:08:40.881 INFO:teuthology.orchestra.run.smithi145.stdout:1355 read 0x480e000 thru 0x4813d6f (0x5d70 bytes) 2024-09-14T07:08:40.882 INFO:teuthology.orchestra.run.smithi145.stdout:1356 read 0xc6df400 thru 0xc6e0bdd (0x17de bytes) 2024-09-14T07:08:40.882 INFO:teuthology.orchestra.run.smithi145.stdout:1357 punch from 0xa767e00 to 0xa774c00, (0xce00 bytes) 2024-09-14T07:08:40.888 INFO:teuthology.orchestra.run.smithi145.stdout:1358 punch from 0xc0c8c00 to 0xc0c9a00, (0xe00 bytes) 2024-09-14T07:08:40.893 INFO:teuthology.orchestra.run.smithi145.stdout:1359 write 0xaa1d000 thru 0xaa1fbec (0x2bed bytes) 2024-09-14T07:08:40.894 INFO:teuthology.orchestra.run.smithi145.stdout:1360 write 0xcadca00 thru 0xcae7bb7 (0xb1b8 bytes) 2024-09-14T07:08:40.894 INFO:teuthology.orchestra.run.smithi145.stdout:1363 write 0xa2b5000 thru 0xa2b6a70 (0x1a71 bytes) 2024-09-14T07:08:40.895 INFO:teuthology.orchestra.run.smithi145.stdout:1364 read 0xcff4e00 thru 0xcff5cb0 (0xeb1 bytes) 2024-09-14T07:08:40.896 INFO:teuthology.orchestra.run.smithi145.stdout:1365 punch from 0xcc23600 to 0xcc2c200, (0x8c00 bytes) 2024-09-14T07:08:40.908 INFO:teuthology.orchestra.run.smithi145.stdout:1367 punch from 0xaa24600 to 0xaa29c00, (0x5600 bytes) 2024-09-14T07:08:40.909 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:40.905+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:40.910 INFO:teuthology.orchestra.run.smithi145.stdout:1368 write 0x1a06c00 thru 0x1a12f73 (0xc374 bytes) 2024-09-14T07:08:40.911 INFO:teuthology.orchestra.run.smithi145.stdout:1370 write 0x78d0000 thru 0x78d68e4 (0x68e5 bytes) 2024-09-14T07:08:40.912 INFO:teuthology.orchestra.run.smithi145.stdout:1372 write 0x829ec00 thru 0x82ad37a (0xe77b bytes) 2024-09-14T07:08:40.914 INFO:teuthology.orchestra.run.smithi145.stdout:1375 write 0xde9f600 thru 0xdead733 (0xe134 bytes) 2024-09-14T07:08:40.914 INFO:teuthology.orchestra.run.smithi145.stdout:1379 write 0xd8c6000 thru 0xd8cc9b7 (0x69b8 bytes) 2024-09-14T07:08:40.914 INFO:teuthology.orchestra.run.smithi145.stdout:1380 punch from 0xb355800 to 0xb357800, (0x2000 bytes) 2024-09-14T07:08:40.955 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:40.955 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:40.963 INFO:teuthology.orchestra.run.smithi145.stdout:1381 read 0x92b600 thru 0x937df3 (0xc7f4 bytes) 2024-09-14T07:08:40.965 INFO:teuthology.orchestra.run.smithi145.stdout:1382 trunc from 0xdf9b200 to 0xdf6c800 2024-09-14T07:08:40.970 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:41.009 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:41.005+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:41.019 INFO:teuthology.orchestra.run.smithi145.stdout:1385 write 0xe489000 thru 0xe489a1c (0xa1d bytes) 2024-09-14T07:08:41.022 INFO:teuthology.orchestra.run.smithi145.stdout:1386 punch from 0x6a4f400 to 0x6a58600, (0x9200 bytes) 2024-09-14T07:08:41.055 INFO:teuthology.orchestra.run.smithi145.stdout:1388 trunc from 0xe489c00 to 0xa970600 2024-09-14T07:08:41.110 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:41.105+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:41.135 INFO:teuthology.orchestra.run.smithi145.stdout:1389 punch from 0x224f600 to 0x2257000, (0x7a00 bytes) 2024-09-14T07:08:41.138 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:41.138 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:41.153 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:08:41.156 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:41.190 INFO:teuthology.orchestra.run.smithi145.stdout:1390 trunc from 0xa970600 to 0x5ceaa00 2024-09-14T07:08:41.219 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:41.217+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:41.241 INFO:teuthology.orchestra.run.smithi145.stdout:1391 flatten 2024-09-14T07:08:41.263 INFO:teuthology.orchestra.run.smithi145.stdout:1392 read 0x5759600 thru 0x575cb62 (0x3563 bytes) 2024-09-14T07:08:41.264 INFO:teuthology.orchestra.run.smithi145.stdout:1395 write 0xf38400 thru 0xf3f511 (0x7112 bytes) 2024-09-14T07:08:41.292 INFO:teuthology.orchestra.run.smithi145.stdout:1397 write 0x6630c00 thru 0x663a45b (0x985c bytes) 2024-09-14T07:08:41.296 INFO:teuthology.orchestra.run.smithi145.stdout:1398 read 0x6e1000 thru 0x6e3486 (0x2487 bytes) 2024-09-14T07:08:41.297 INFO:teuthology.orchestra.run.smithi145.stdout:1399 write 0x5434a00 thru 0x54356d8 (0xcd9 bytes) 2024-09-14T07:08:41.298 INFO:teuthology.orchestra.run.smithi145.stdout:1400 write 0x2d0de00 thru 0x2d1d77a (0xf97b bytes) 2024-09-14T07:08:41.299 INFO:teuthology.orchestra.run.smithi145.stdout:1401 trunc from 0x663a600 to 0x6640a00 2024-09-14T07:08:41.312 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:41.309+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:41.334 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:41.334 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:41.349 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:41.354 INFO:teuthology.orchestra.run.smithi145.stdout:1402 write 0x6738c00 thru 0x673e079 (0x547a bytes) 2024-09-14T07:08:41.383 INFO:teuthology.orchestra.run.smithi145.stdout:1404 write 0x78bd000 thru 0x78c5e5d (0x8e5e bytes) 2024-09-14T07:08:41.387 INFO:teuthology.orchestra.run.smithi145.stdout:1406 punch from 0x2962200 to 0x2963e00, (0x1c00 bytes) 2024-09-14T07:08:41.387 INFO:teuthology.orchestra.run.smithi145.stdout:1410 punch from 0x3c3d200 to 0x3c4c200, (0xf000 bytes) 2024-09-14T07:08:41.388 INFO:teuthology.orchestra.run.smithi145.stdout:1411 write 0x77d2800 thru 0x77d2b17 (0x318 bytes) 2024-09-14T07:08:41.389 INFO:teuthology.orchestra.run.smithi145.stdout:1412 punch from 0x7070200 to 0x7075200, (0x5000 bytes) 2024-09-14T07:08:41.409 INFO:teuthology.orchestra.run.smithi145.stdout:1413 write 0x7ee0800 thru 0x7ee6ef1 (0x66f2 bytes) 2024-09-14T07:08:41.411 INFO:teuthology.orchestra.run.smithi145.stdout:1414 read 0x4a30800 thru 0x4a3f79a (0xef9b bytes) 2024-09-14T07:08:41.413 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:41.409+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:41.448 INFO:teuthology.orchestra.run.smithi145.stdout:1417 write 0xba2da00 thru 0xba3bd12 (0xe313 bytes) 2024-09-14T07:08:41.479 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:41.479 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:41.481 INFO:teuthology.orchestra.run.smithi145.stdout:1420 write 0xc67aa00 thru 0xc67f767 (0x4d68 bytes) 2024-09-14T07:08:41.485 INFO:teuthology.orchestra.run.smithi145.stdout:1425 read 0xacd1e00 thru 0xacd6219 (0x441a bytes) 2024-09-14T07:08:41.485 INFO:teuthology.orchestra.run.smithi145.stdout:1426 write 0x6a26000 thru 0x6a289ad (0x29ae bytes) 2024-09-14T07:08:41.486 INFO:teuthology.orchestra.run.smithi145.stdout:1427 write 0x3be3a00 thru 0x3beb2a1 (0x78a2 bytes) 2024-09-14T07:08:41.487 INFO:teuthology.orchestra.run.smithi145.stdout:1428 read 0x40d5800 thru 0x40e0f93 (0xb794 bytes) 2024-09-14T07:08:41.488 INFO:teuthology.orchestra.run.smithi145.stdout:1429 punch from 0x14e400 to 0x15b400, (0xd000 bytes) 2024-09-14T07:08:41.489 INFO:teuthology.orchestra.run.smithi145.stdout:1431 write 0xa1b800 thru 0xa2b778 (0xff79 bytes) 2024-09-14T07:08:41.490 INFO:teuthology.orchestra.run.smithi145.stdout:1432 read 0x66c4e00 thru 0x66c897c (0x3b7d bytes) 2024-09-14T07:08:41.491 INFO:teuthology.orchestra.run.smithi145.stdout:1433 write 0x372400 thru 0x374c31 (0x2832 bytes) 2024-09-14T07:08:41.496 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:41.496 INFO:teuthology.orchestra.run.smithi145.stdout:1435 write 0x1794600 thru 0x1798ede (0x48df bytes) 2024-09-14T07:08:41.496 INFO:teuthology.orchestra.run.smithi145.stdout:1437 read 0xa04be00 thru 0xa05215f (0x6360 bytes) 2024-09-14T07:08:41.497 INFO:teuthology.orchestra.run.smithi145.stdout:1440 read 0x2f4e800 thru 0x2f55e44 (0x7645 bytes) 2024-09-14T07:08:41.498 INFO:teuthology.orchestra.run.smithi145.stdout:1445 read 0x2d0ec00 thru 0x2d1876c (0x9b6d bytes) 2024-09-14T07:08:41.500 INFO:teuthology.orchestra.run.smithi145.stdout:1447 read 0xb16d200 thru 0xb1731ec (0x5fed bytes) 2024-09-14T07:08:41.500 INFO:teuthology.orchestra.run.smithi145.stdout:1449 read 0x9a71400 thru 0x9a7f338 (0xdf39 bytes) 2024-09-14T07:08:41.501 INFO:teuthology.orchestra.run.smithi145.stdout:1453 trunc from 0xc67f800 to 0xc5c7200 2024-09-14T07:08:41.515 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:41.509+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:41.537 INFO:teuthology.orchestra.run.smithi145.stdout:1454 read 0xafb7a00 thru 0xafbe5e5 (0x6be6 bytes) 2024-09-14T07:08:41.570 INFO:teuthology.orchestra.run.smithi145.stdout:1456 write 0xdb0c000 thru 0xdb1a24b (0xe24c bytes) 2024-09-14T07:08:41.593 INFO:teuthology.orchestra.run.smithi145.stdout:1457 trunc from 0xdb1a400 to 0x49ff000 2024-09-14T07:08:41.615 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:41.613+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:41.644 INFO:teuthology.orchestra.run.smithi145.stdout:1458 read 0x4275c00 thru 0x4281019 (0xb41a bytes) 2024-09-14T07:08:41.660 INFO:teuthology.orchestra.run.smithi145.stdout:1459 trunc from 0x49ff000 to 0x9dba200 2024-09-14T07:08:41.669 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:41.669 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:41.684 INFO:teuthology.orchestra.run.smithi145.stdout:1461 read 0x21e2c00 thru 0x21e964b (0x6a4c bytes) 2024-09-14T07:08:41.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:41.685 INFO:teuthology.orchestra.run.smithi145.stdout:1462 read 0x7401800 thru 0x7404fbd (0x37be bytes) 2024-09-14T07:08:41.686 INFO:teuthology.orchestra.run.smithi145.stdout:1463 read 0x7539e00 thru 0x75473d4 (0xd5d5 bytes) 2024-09-14T07:08:41.717 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:41.713+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:41.732 INFO:teuthology.orchestra.run.smithi145.stdout:1464 trunc from 0x9dba200 to 0x2459800 2024-09-14T07:08:41.763 INFO:teuthology.orchestra.run.smithi145.stdout:1475 read 0x1546600 thru 0x154ddcc (0x77cd bytes) 2024-09-14T07:08:41.767 INFO:teuthology.orchestra.run.smithi145.stdout:1476 read 0x79be00 thru 0x79c6f1 (0x8f2 bytes) 2024-09-14T07:08:41.768 INFO:teuthology.orchestra.run.smithi145.stdout:1477 punch from 0x1debc00 to 0x1df9a00, (0xde00 bytes) 2024-09-14T07:08:41.781 INFO:teuthology.orchestra.run.smithi145.stdout:1478 trunc from 0x2459800 to 0x7926400 2024-09-14T07:08:41.818 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:41.813+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:41.826 INFO:teuthology.orchestra.run.smithi145.stdout:1480 write 0xaa26a00 thru 0xaa33d05 (0xd306 bytes) 2024-09-14T07:08:41.827 INFO:teuthology.orchestra.run.smithi145.stdout:1482 read 0x3c4d400 thru 0x3c58fbb (0xbbbc bytes) 2024-09-14T07:08:41.834 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:41.834 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:41.846 INFO:teuthology.orchestra.run.smithi145.stdout:1485 trunc from 0xaa33e00 to 0x3841c00 2024-09-14T07:08:41.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:41.893 INFO:teuthology.orchestra.run.smithi145.stdout:1486 punch from 0x2bffc00 to 0x2c0e800, (0xec00 bytes) 2024-09-14T07:08:41.896 INFO:teuthology.orchestra.run.smithi145.stdout:1489 read 0x354d800 thru 0x3558f31 (0xb732 bytes) 2024-09-14T07:08:41.898 INFO:teuthology.orchestra.run.smithi145.stdout:1493 read 0x232f200 thru 0x23303fe (0x11ff bytes) 2024-09-14T07:08:41.899 INFO:teuthology.orchestra.run.smithi145.stdout:1494 write 0x153ac00 thru 0x1546f2a (0xc32b bytes) 2024-09-14T07:08:41.902 INFO:teuthology.orchestra.run.smithi145.stdout:1495 trunc from 0x3841c00 to 0x4713800 2024-09-14T07:08:41.923 INFO:teuthology.orchestra.run.smithi145.stdout:1496 write 0x2aeec00 thru 0x2afa87d (0xbc7e bytes) 2024-09-14T07:08:41.935 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:41.933+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:41.973 INFO:teuthology.orchestra.run.smithi145.stdout:1497 write 0xca1b800 thru 0xca1d1f4 (0x19f5 bytes) 2024-09-14T07:08:41.975 INFO:teuthology.orchestra.run.smithi145.stdout:1499 read 0xa052e00 thru 0xa062d19 (0xff1a bytes) 2024-09-14T07:08:41.976 INFO:teuthology.orchestra.run.smithi145.stdout:1501 read 0x8935600 thru 0x8937467 (0x1e68 bytes) 2024-09-14T07:08:42.001 INFO:teuthology.orchestra.run.smithi145.stdout:1504 write 0xdde2000 thru 0xddef612 (0xd613 bytes) 2024-09-14T07:08:42.002 INFO:teuthology.orchestra.run.smithi145.stdout:1505 write 0x312ae00 thru 0x312e393 (0x3594 bytes) 2024-09-14T07:08:42.003 INFO:teuthology.orchestra.run.smithi145.stdout:1506 write 0xac79e00 thru 0xac873d9 (0xd5da bytes) 2024-09-14T07:08:42.003 INFO:teuthology.orchestra.run.smithi145.stdout:1508 read 0x4fc7200 thru 0x4fc82b2 (0x10b3 bytes) 2024-09-14T07:08:42.004 INFO:teuthology.orchestra.run.smithi145.stdout:1510 punch from 0x3acc200 to 0x3ad9600, (0xd400 bytes) 2024-09-14T07:08:42.005 INFO:teuthology.orchestra.run.smithi145.stdout:1511 write 0xad17c00 thru 0xad1be9c (0x429d bytes) 2024-09-14T07:08:42.005 INFO:teuthology.orchestra.run.smithi145.stdout:1515 write 0x6f43600 thru 0x6f4911f (0x5b20 bytes) 2024-09-14T07:08:42.006 INFO:teuthology.orchestra.run.smithi145.stdout:1516 punch from 0x6f62200 to 0x6f6de00, (0xbc00 bytes) 2024-09-14T07:08:42.007 INFO:teuthology.orchestra.run.smithi145.stdout:1517 write 0x759dc00 thru 0x75ad359 (0xf75a bytes) 2024-09-14T07:08:42.009 INFO:teuthology.orchestra.run.smithi145.stdout:1518 write 0x6fa2800 thru 0x6fae8a2 (0xc0a3 bytes) 2024-09-14T07:08:42.010 INFO:teuthology.orchestra.run.smithi145.stdout:1520 write 0xa662600 thru 0xa66fb77 (0xd578 bytes) 2024-09-14T07:08:42.010 INFO:teuthology.orchestra.run.smithi145.stdout:1528 punch from 0x4fd800 to 0x501200, (0x3a00 bytes) 2024-09-14T07:08:42.012 INFO:teuthology.orchestra.run.smithi145.stdout:1529 write 0x9f8c000 thru 0x9f99eb5 (0xdeb6 bytes) 2024-09-14T07:08:42.019 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:42.017+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:42.025 INFO:teuthology.orchestra.run.smithi145.stdout:1530 trunc from 0xddef800 to 0x995ac00 2024-09-14T07:08:42.034 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:42.034 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:42.052 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:42.055 INFO:teuthology.orchestra.run.smithi145.stdout:1531 punch from 0x579ea00 to 0x57a5a00, (0x7000 bytes) 2024-09-14T07:08:42.069 INFO:teuthology.orchestra.run.smithi145.stdout:1533 trunc from 0x995ac00 to 0x28e2000 2024-09-14T07:08:42.118 INFO:teuthology.orchestra.run.smithi145.stdout:1536 write 0x71cee00 thru 0x71d5237 (0x6438 bytes) 2024-09-14T07:08:42.121 INFO:teuthology.orchestra.run.smithi145.stdout:1539 punch from 0x3c89a00 to 0x3c95c00, (0xc200 bytes) 2024-09-14T07:08:42.121 INFO:teuthology.orchestra.run.smithi145.stdout:1541 punch from 0x6b6ae00 to 0x6b78e00, (0xe000 bytes) 2024-09-14T07:08:42.121 INFO:teuthology.orchestra.run.smithi145.stdout:1542 write 0x4d00000 thru 0x4d0b3c9 (0xb3ca bytes) 2024-09-14T07:08:42.123 INFO:teuthology.orchestra.run.smithi145.stdout:1543 trunc from 0x71d5400 to 0x6639400 2024-09-14T07:08:42.139 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:42.137+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:42.142 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 2, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:42.143 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:42.143 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T07:08:42.143 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:08:42.143 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:08:42.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T07:08:42.144 INFO:teuthology.orchestra.run.smithi145.stdout:1544 write 0x5a89000 thru 0x5a961d9 (0xd1da bytes) 2024-09-14T07:08:42.159 INFO:teuthology.orchestra.run.smithi145.stdout:1546 trunc from 0x6639400 to 0xdb9d200 2024-09-14T07:08:42.197 INFO:teuthology.orchestra.run.smithi145.stdout:1547 trunc from 0xdb9d200 to 0x5507800 2024-09-14T07:08:42.219 INFO:teuthology.orchestra.run.smithi145.stdout:1549 read 0x1655e00 thru 0x1656d66 (0xf67 bytes) 2024-09-14T07:08:42.221 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:42.217+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:42.221 INFO:teuthology.orchestra.run.smithi145.stdout:1550 read 0xe1000 thru 0xe2c95 (0x1c96 bytes) 2024-09-14T07:08:42.226 INFO:teuthology.orchestra.run.smithi145.stdout:1551 punch from 0x445400 to 0x453200, (0xde00 bytes) 2024-09-14T07:08:42.227 INFO:teuthology.orchestra.run.smithi145.stdout:1552 read 0x119c200 thru 0x11a0d88 (0x4b89 bytes) 2024-09-14T07:08:42.227 INFO:teuthology.orchestra.run.smithi145.stdout:1559 read 0x46e5800 thru 0x46ef124 (0x9925 bytes) 2024-09-14T07:08:42.228 INFO:teuthology.orchestra.run.smithi145.stdout:1564 punch from 0x274c000 to 0x274e800, (0x2800 bytes) 2024-09-14T07:08:42.228 INFO:teuthology.orchestra.run.smithi145.stdout:1565 read 0x61f000 thru 0x6209af (0x19b0 bytes) 2024-09-14T07:08:42.240 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:42.240 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:42.243 INFO:teuthology.orchestra.run.smithi145.stdout:1566 trunc from 0x5507800 to 0xcfdf600 2024-09-14T07:08:42.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:42.273 INFO:teuthology.orchestra.run.smithi145.stdout:1568 trunc from 0xcfdf600 to 0x476b200 2024-09-14T07:08:42.277 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:42.277 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "" 2024-09-14T07:08:42.277 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:42.298 INFO:teuthology.orchestra.run.smithi145.stdout:1572 write 0x2eab600 thru 0x2eb70d3 (0xbad4 bytes) 2024-09-14T07:08:42.300 INFO:teuthology.orchestra.run.smithi145.stdout:1573 punch from 0x257f600 to 0x258f400, (0xfe00 bytes) 2024-09-14T07:08:42.316 INFO:teuthology.orchestra.run.smithi145.stdout:1574 trunc from 0x476b200 to 0xc877400 2024-09-14T07:08:42.336 INFO:teuthology.orchestra.run.smithi145.stdout:1575 trunc from 0xc877400 to 0x82e4600 2024-09-14T07:08:42.339 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:42.337+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:42.356 INFO:teuthology.orchestra.run.smithi145.stdout:1579 clone 12 order 24 su 65536 sc 15 2024-09-14T07:08:42.396 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:42.396 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:42.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:42.422 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:42.417+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:42.523 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:42.517+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:42.583 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:42.583 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:42.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:42.623 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:42.621+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:42.719 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:42.720 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:42.724 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:42.721+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:42.735 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:42.825 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:42.821+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:42.863 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:42.863 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:42.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:42.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:42.967 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:42.965+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:43.014 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:43.028 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:43.025+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:43.031 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:43.031 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:43.048 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:43.056 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:43.056 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:43.057 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:43.057 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:43.057 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:43.069 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:43.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:43.090 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:43.112 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:43.112 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:43.128 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:43.125+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:43.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:08:43.216 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1538 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994537698999999, 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054129517000000002, 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1538 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:08:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220949+0000", 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-06 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220959+0000", 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0932e-05 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220965+0000", 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.49e-06 2024-09-14T07:08:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274976+0000", 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054011221999999998 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275019+0000", 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2889000000000002e-05 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275076+0000", 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6455000000000001e-05 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1539 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994510983000001, 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054229843, 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1539 2024-09-14T07:08:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220976+0000", 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-06 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1069999999999999e-06 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220988+0000", 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0119999999999996e-06 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275106+0000", 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054118053999999999 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275149+0000", 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2830000000000002e-05 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275203+0000", 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4044e-05 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1540 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994488666999999, 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05433267, 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1540 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220996+0000", 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220998+0000", 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9759999999999998e-06 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221009+0000", 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.242e-06 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275233+0000", 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054223461000000001 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275275+0000", 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2726999999999999e-05 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275328+0000", 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2332e-05 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1541 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994466470999999, 2024-09-14T07:08:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054437230000000003, 2024-09-14T07:08:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1541 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221018+0000", 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2e-07 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221020+0000", 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3530000000000002e-06 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221026+0000", 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3110000000000002e-06 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221031+0000", 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275358+0000", 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054326722000000001 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275401+0000", 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2849000000000003e-05 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275454+0000", 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3844000000000002e-05 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1542 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994446057000001, 2024-09-14T07:08:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054541385999999997, 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1542 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3700000000000001e-07 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221040+0000", 2024-09-14T07:08:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8059999999999999e-06 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221046+0000", 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1009999999999997e-06 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221051+0000", 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6940000000000003e-06 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275484+0000", 2024-09-14T07:08:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054433003000000001 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275527+0000", 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3031999999999997e-05 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275579+0000", 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2413000000000001e-05 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1543 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994426301000001, 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054648398000000001, 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1543 2024-09-14T07:08:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221058+0000", 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4299999999999999e-07 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221060+0000", 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7990000000000001e-06 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221066+0000", 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221070+0000", 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7010000000000004e-06 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275609+0000", 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054538968 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275652+0000", 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2864000000000002e-05 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275706+0000", 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3597000000000002e-05 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1545 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994406635000001, 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054855590000000003, 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1545 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4499999999999998e-07 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221085+0000", 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0330000000000008e-06 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221092+0000", 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.793e-06 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221098+0000", 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3090000000000003e-06 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275744+0000", 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054646404000000003 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275821+0000", 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6793999999999997e-05 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275933+0000", 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111912 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1583 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 524288~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994378472000001, 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055023028000000002, 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1583 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221106+0000", 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221108+0000", 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0229999999999999e-06 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221121+0000", 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3645000000000001e-05 2024-09-14T07:08:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221127+0000", 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.541e-06 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275735+0000", 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054608372000000002 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275817+0000", 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2083999999999995e-05 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276128+0000", 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031090000000000002 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1585 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994348637999998, 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055153874999999998, 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1585 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:43.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221136+0000", 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7300000000000001e-07 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221138+0000", 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221144+0000", 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5100000000000004e-06 2024-09-14T07:08:43.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221149+0000", 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.843e-06 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276165+0000", 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055015677999999998 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276231+0000", 2024-09-14T07:08:43.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5959999999999999e-05 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276289+0000", 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8100000000000003e-05 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1586 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994325023999998, 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055262004000000003, 2024-09-14T07:08:43.256 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1586 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221166+0000", 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3180000000000001e-06 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221173+0000", 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221178+0000", 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2050000000000001e-06 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276321+0000", 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055142663000000001 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276364+0000", 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3615000000000002e-05 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276421+0000", 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6453000000000003e-05 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1588 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994297787000001, 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055364653999999999, 2024-09-14T07:08:43.260 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1588 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221189+0000", 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3269999999999999e-06 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221195+0000", 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2169999999999999e-06 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221200+0000", 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276451+0000", 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055251452999999999 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276497+0000", 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5114000000000002e-05 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276551+0000", 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4079999999999997e-05 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1589 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994277123, 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055473737000000002, 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1589 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:43.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221209+0000", 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9269999999999999e-06 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221215+0000", 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3069999999999999e-06 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221220+0000", 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6369999999999998e-06 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276582+0000", 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055361856000000001 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276625+0000", 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2803000000000003e-05 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276680+0000", 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5789999999999999e-05 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1591 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.994256762999999, 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055584770999999998, 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.269 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1591 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222763+0000", 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001535914 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222789+0000", 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6194e-05 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222862+0000", 2024-09-14T07:08:43.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2529999999999998e-05 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222953+0000", 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1185000000000002e-05 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276713+0000", 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053760371000000001 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276758+0000", 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4932000000000003e-05 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276812+0000", 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3644999999999999e-05 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1593 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 6225920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.273 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.992486088, 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054086803000000003, 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1593 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223000+0000", 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.289e-06 2024-09-14T07:08:43.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223007+0000", 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.058e-06 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223013+0000", 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5779999999999998e-06 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276973+0000", 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053960263000000001 2024-09-14T07:08:43.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277028+0000", 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000002e-05 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277084+0000", 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6357999999999998e-05 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1594 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.992462223, 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054193419, 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1594 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223035+0000", 2024-09-14T07:08:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2994e-05 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223043+0000", 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6539999999999999e-06 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223048+0000", 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6149999999999996e-06 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277114+0000", 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054066256 2024-09-14T07:08:43.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277157+0000", 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2629000000000001e-05 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277215+0000", 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7803999999999997e-05 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.281 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1598 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.991842668, 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054092236000000002, 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1598 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224174+0000", 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000533265 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224179+0000", 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2420000000000002e-06 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224190+0000", 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-05 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224197+0000", 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3549999999999999e-06 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277635+0000", 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053437314999999999 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277679+0000", 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4628e-05 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277733+0000", 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4131000000000001e-05 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.285 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.915177031999999, 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:08:43.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:08:43.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 25.870186726, 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.296 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.568254270000001, 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:08:43.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:08:43.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.246926536, 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:43.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:43.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:08:43.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:08:43.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:08:43.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:43.307 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:43.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:43.309 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:43.305+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:43.410 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:43.405+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:43.447 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:43.447 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:43.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:43.511 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:43.509+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:43.611 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:43.609+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:43.626 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:43.626 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:43.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:43.712 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:43.709+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:43.750 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 71.507889148999993, 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.811 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:43.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:43.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:43.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1502 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.587524789, 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042080580999999999, 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1502 2024-09-14T07:08:43.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220868+0000", 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220869+0000", 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2300000000000001e-06 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220888+0000", 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8607e-05 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220907+0000", 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9422000000000001e-05 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220922+0000", 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4375999999999999e-05 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262777+0000", 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855537999999998 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262833+0000", 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5729000000000001e-05 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262949+0000", 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115679 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1503 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.587449976999999, 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042296750000000001, 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1503 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220945+0000", 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.013e-06 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220951+0000", 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.806e-06 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220970+0000", 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8649999999999999e-05 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3964999999999999e-05 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262989+0000", 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042005845 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263116+0000", 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012688 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263240+0000", 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012359099999999999 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1504 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:43.826 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.587390852999999, 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042493942999999999, 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1504 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.063e-06 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221011+0000", 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8059999999999997e-06 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221030+0000", 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8652999999999999e-05 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221044+0000", 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3879999999999999e-05 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263275+0000", 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042231869999999998 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263365+0000", 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9813000000000002e-05 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263496+0000", 2024-09-14T07:08:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000130858 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1634 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.584196791, 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041961863000000002, 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1634 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224196+0000", 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224198+0000", 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.77e-06 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224205+0000", 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6429999999999998e-06 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224240+0000", 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5714999999999999e-05 2024-09-14T07:08:43.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224555+0000", 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031487300000000002 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265911+0000", 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041355780000000002 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266040+0000", 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012856099999999999 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266158+0000", 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118521 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1636 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.583663166000001, 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042143574000000003, 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1636 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:43.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224731+0000", 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5600000000000001e-06 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224738+0000", 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224756+0000", 2024-09-14T07:08:43.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8153999999999999e-05 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224770+0000", 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4168999999999999e-05 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266618+0000", 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041847662000000001 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266730+0000", 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000111993 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266873+0000", 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014374400000000001 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1643 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 8978432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:43.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582582130999999, 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042103224000000002, 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1643 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225811+0000", 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225812+0000", 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.207e-06 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225818+0000", 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225836+0000", 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8221999999999999e-05 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225849+0000", 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.358e-05 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267705+0000", 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041855723999999997 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267805+0000", 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9437000000000004e-05 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267914+0000", 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109554 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1644 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582523902999998, 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042307838, 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1644 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225870+0000", 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2270000000000001e-06 2024-09-14T07:08:43.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225876+0000", 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2569999999999998e-06 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225894+0000", 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8385000000000001e-05 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225936+0000", 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2159000000000001e-05 2024-09-14T07:08:43.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267952+0000", 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04201585 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268067+0000", 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114985 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268177+0000", 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109975 2024-09-14T07:08:43.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1645 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582435526000001, 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042484558999999998, 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1645 2024-09-14T07:08:43.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225959+0000", 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:43.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225978+0000", 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9466e-05 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225998+0000", 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9315999999999999e-05 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226013+0000", 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:43.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268213+0000", 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200410000000001 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268324+0000", 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011123799999999999 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268442+0000", 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117889 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.851 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1646 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582359147999998, 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042595283999999997, 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1646 2024-09-14T07:08:43.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226035+0000", 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5060000000000001e-06 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226042+0000", 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4370000000000004e-06 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226060+0000", 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8266000000000001e-05 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226074+0000", 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4029000000000001e-05 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042405023999999999 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268551+0000", 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1973999999999999e-05 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268629+0000", 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8047999999999995e-05 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1647 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12910592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582301114, 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042800796000000002, 2024-09-14T07:08:43.856 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1647 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226093+0000", 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.46e-06 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226099+0000", 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3530000000000002e-06 2024-09-14T07:08:43.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226118+0000", 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9055999999999998e-05 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226132+0000", 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268673+0000", 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042541272999999998 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268775+0000", 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102302 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268893+0000", 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117539 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.860 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1648 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582241972999999, 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042904468000000001, 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1648 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226152+0000", 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.234e-06 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226157+0000", 2024-09-14T07:08:43.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2000000000000002e-06 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226175+0000", 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7059e-05 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226204+0000", 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9536e-05 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268938+0000", 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042733698000000001 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268992+0000", 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4693e-05 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269056+0000", 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3047999999999997e-05 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1649 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.864 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582167765000001, 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043088794999999999, 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1649 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226227+0000", 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.33e-06 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226232+0000", 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.733e-06 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226251+0000", 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8816000000000001e-05 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226264+0000", 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-05 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269097+0000", 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0428324 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269197+0000", 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100419 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269314+0000", 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116897 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1651 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582108916999999, 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043199128000000003, 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1651 2024-09-14T07:08:43.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226285+0000", 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.237e-06 2024-09-14T07:08:43.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226291+0000", 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1789999999999999e-06 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226311+0000", 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0656999999999999e-05 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226327+0000", 2024-09-14T07:08:43.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5948999999999999e-05 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269352+0000", 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043024830999999999 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269419+0000", 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7281999999999997e-05 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269483+0000", 2024-09-14T07:08:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3992999999999995e-05 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1700 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 32768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.582045589, 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043417208999999998, 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1700 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226349+0000", 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3629999999999999e-06 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226354+0000", 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226372+0000", 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7612999999999999e-05 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226386+0000", 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4146e-05 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269525+0000", 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043139380999999997 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269631+0000", 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105574 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269765+0000", 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133759 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1702 3.13 3:cfeabdd5:::rbd_data.11616dafc67f.000000000000000f:9 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:43.877 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.581957168999999, 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043560261000000003, 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1702 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226437+0000", 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.567e-06 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226457+0000", 2024-09-14T07:08:43.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395999999999999e-05 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226477+0000", 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9998999999999999e-05 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226491+0000", 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4503e-05 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269835+0000", 2024-09-14T07:08:43.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043343995000000003 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269914+0000", 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9018999999999997e-05 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269996+0000", 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1781999999999997e-05 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.881 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1740 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.581880275, 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043673828999999997, 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1740 2024-09-14T07:08:43.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226514+0000", 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4869999999999999e-06 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226520+0000", 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.976e-06 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226538+0000", 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7972999999999999e-05 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226551+0000", 2024-09-14T07:08:43.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3142999999999999e-05 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270041+0000", 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043490174999999999 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270101+0000", 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9515999999999997e-05 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270187+0000", 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5558999999999999e-05 2024-09-14T07:08:43.885 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.517106013999999, 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:43.886 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:08:43.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4803.0:1903 3.1f e43/21 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head v 43'21, mlcod=42'19)", 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.499945194999999, 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.057064131999999997, 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308449+0000", 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.279e-06 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324156+0000", 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015707199000000002 2024-09-14T07:08:43.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324195+0000", 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9070999999999999e-05 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324217+0000", 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1752000000000001e-05 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324333+0000", 2024-09-14T07:08:43.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011607000000000001 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324359+0000", 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6296e-05 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365477+0000", 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041117065000000001 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365512+0000", 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-05 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4703.0:530 3.f e46/21 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head v 46'75, mlcod=45'73)", 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.94600625, 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.075870370000000006, 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:43.896 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862389+0000", 2024-09-14T07:08:43.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6579999999999998e-06 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870883+0000", 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084939100000000003 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870916+0000", 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2428000000000002e-05 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:43.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870931+0000", 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5656000000000001e-05 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871022+0000", 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1063000000000005e-05 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871047+0000", 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4125e-05 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938212+0000", 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067164920000000003 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938257+0000", 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5609999999999999e-05 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:43.900 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:43.901 INFO:teuthology.orchestra.run.smithi145.stdout:1582 read 0x5a06000 thru 0x5a0cb3f (0x6b40 bytes) 2024-09-14T07:08:43.902 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:43.902 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:43.897+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:43.909 INFO:teuthology.orchestra.run.smithi145.stdout:1585 write 0xd278400 thru 0xd285f4a (0xdb4b bytes) 2024-09-14T07:08:43.912 INFO:teuthology.orchestra.run.smithi145.stdout:1591 write 0x2265e00 thru 0x226e0c0 (0x82c1 bytes) 2024-09-14T07:08:43.926 INFO:teuthology.orchestra.run.smithi145.stdout:1592 trunc from 0xd286000 to 0x8422000 2024-09-14T07:08:44.003 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:43.997+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:44.103 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:44.101+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:44.108 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:44.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:44.108 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:44.108 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:44.124 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:44.138 INFO:teuthology.orchestra.run.smithi145.stdout:1594 trunc from 0x8422000 to 0xdf03200 2024-09-14T07:08:44.155 INFO:teuthology.orchestra.run.smithi145.stdout:1595 read 0xbeb0600 thru 0xbebe295 (0xdc96 bytes) 2024-09-14T07:08:44.159 INFO:teuthology.orchestra.run.smithi145.stdout:1597 read 0x103c800 thru 0x104007e (0x387f bytes) 2024-09-14T07:08:44.160 INFO:teuthology.orchestra.run.smithi145.stdout:1599 read 0xde66400 thru 0xde71b3f (0xb740 bytes) 2024-09-14T07:08:44.162 INFO:teuthology.orchestra.run.smithi145.stdout:1601 write 0xd25c600 thru 0xd2645ee (0x7fef bytes) 2024-09-14T07:08:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:08:44.235 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:44.233+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:44.238 INFO:teuthology.orchestra.run.smithi145.stdout:1603 trunc from 0xdf03200 to 0x3088000 2024-09-14T07:08:44.288 INFO:teuthology.orchestra.run.smithi145.stdout:1604 read 0x1701c00 thru 0x1704c70 (0x3071 bytes) 2024-09-14T07:08:44.291 INFO:teuthology.orchestra.run.smithi145.stdout:1607 read 0x1c0ba00 thru 0x1c0e326 (0x2927 bytes) 2024-09-14T07:08:44.292 INFO:teuthology.orchestra.run.smithi145.stdout:1608 punch from 0xdff000 to 0xe04a00, (0x5a00 bytes) 2024-09-14T07:08:44.292 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:44.292 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:44.292 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:44.292 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:44.292 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:44.293 INFO:teuthology.orchestra.run.smithi145.stdout:1610 punch from 0x1e2d200 to 0x1e32800, (0x5600 bytes) 2024-09-14T07:08:44.293 INFO:teuthology.orchestra.run.smithi145.stdout:1611 read 0x2ae5600 thru 0x2ae875a (0x315b bytes) 2024-09-14T07:08:44.293 INFO:teuthology.orchestra.run.smithi145.stdout:1615 punch from 0x149ca00 to 0x149d600, (0xc00 bytes) 2024-09-14T07:08:44.293 INFO:teuthology.orchestra.run.smithi145.stdout:1617 read 0x1130e00 thru 0x11320f4 (0x12f5 bytes) 2024-09-14T07:08:44.295 INFO:teuthology.orchestra.run.smithi145.stdout:1618 read 0x1f2a800 thru 0x1f35907 (0xb108 bytes) 2024-09-14T07:08:44.305 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:44.301+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:44.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:44.311 INFO:teuthology.orchestra.run.smithi145.stdout:1619 trunc from 0x3088000 to 0x9083e00 2024-09-14T07:08:44.337 INFO:teuthology.orchestra.run.smithi145.stdout:1621 write 0x49d6000 thru 0x49dfe35 (0x9e36 bytes) 2024-09-14T07:08:44.340 INFO:teuthology.orchestra.run.smithi145.stdout:1622 flatten 2024-09-14T07:08:44.371 INFO:teuthology.orchestra.run.smithi145.stdout:1624 write 0x8d31a00 thru 0x8d33b1e (0x211f bytes) 2024-09-14T07:08:44.405 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:44.401+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:44.422 INFO:teuthology.orchestra.run.smithi145.stdout:1625 write 0xeb31000 thru 0xeb3dec2 (0xcec3 bytes) 2024-09-14T07:08:44.427 INFO:teuthology.orchestra.run.smithi145.stdout:1626 write 0x8197c00 thru 0x81a34d9 (0xb8da bytes) 2024-09-14T07:08:44.428 INFO:teuthology.orchestra.run.smithi145.stdout:1627 punch from 0x9adfc00 to 0x9ae0e00, (0x1200 bytes) 2024-09-14T07:08:44.429 INFO:teuthology.orchestra.run.smithi145.stdout:1628 write 0xb1f1000 thru 0xb1fe3c5 (0xd3c6 bytes) 2024-09-14T07:08:44.430 INFO:teuthology.orchestra.run.smithi145.stdout:1629 punch from 0xcfc2200 to 0xcfd1000, (0xee00 bytes) 2024-09-14T07:08:44.431 INFO:teuthology.orchestra.run.smithi145.stdout:1631 punch from 0x2397600 to 0x239f600, (0x8000 bytes) 2024-09-14T07:08:44.431 INFO:teuthology.orchestra.run.smithi145.stdout:1632 read 0x7d4ae00 thru 0x7d59fbf (0xf1c0 bytes) 2024-09-14T07:08:44.437 INFO:teuthology.orchestra.run.smithi145.stdout:1634 read 0x27f800 thru 0x2800bc (0x8bd bytes) 2024-09-14T07:08:44.438 INFO:teuthology.orchestra.run.smithi145.stdout:1636 write 0x4b9dc00 thru 0x4b9f92c (0x1d2d bytes) 2024-09-14T07:08:44.439 INFO:teuthology.orchestra.run.smithi145.stdout:1638 write 0x56e1800 thru 0x56e22a6 (0xaa7 bytes) 2024-09-14T07:08:44.440 INFO:teuthology.orchestra.run.smithi145.stdout:1639 read 0x2f4800 thru 0x3002cd (0xbace bytes) 2024-09-14T07:08:44.441 INFO:teuthology.orchestra.run.smithi145.stdout:1640 read 0x17c6800 thru 0x17d198a (0xb18b bytes) 2024-09-14T07:08:44.443 INFO:teuthology.orchestra.run.smithi145.stdout:1641 write 0x7aeba00 thru 0x7af26b7 (0x6cb8 bytes) 2024-09-14T07:08:44.444 INFO:teuthology.orchestra.run.smithi145.stdout:1643 read 0xd61a200 thru 0xd61c91f (0x2720 bytes) 2024-09-14T07:08:44.445 INFO:teuthology.orchestra.run.smithi145.stdout:1646 read 0x40ed000 thru 0x40ed48c (0x48d bytes) 2024-09-14T07:08:44.446 INFO:teuthology.orchestra.run.smithi145.stdout:1647 read 0x9fbae00 thru 0x9fbb15e (0x35f bytes) 2024-09-14T07:08:44.447 INFO:teuthology.orchestra.run.smithi145.stdout:1648 punch from 0xe632a00 to 0xe636c00, (0x4200 bytes) 2024-09-14T07:08:44.448 INFO:teuthology.orchestra.run.smithi145.stdout:1649 read 0x6983a00 thru 0x69904e1 (0xcae2 bytes) 2024-09-14T07:08:44.453 INFO:teuthology.orchestra.run.smithi145.stdout:1650 punch from 0x6647600 to 0x6651800, (0xa200 bytes) 2024-09-14T07:08:44.453 INFO:teuthology.orchestra.run.smithi145.stdout:1652 write 0x9c88600 thru 0x9c88c20 (0x621 bytes) 2024-09-14T07:08:44.454 INFO:teuthology.orchestra.run.smithi145.stdout:1655 read 0xcbfe600 thru 0xcc0cdbd (0xe7be bytes) 2024-09-14T07:08:44.455 INFO:teuthology.orchestra.run.smithi145.stdout:1656 read 0x859e000 thru 0x85a78a6 (0x98a7 bytes) 2024-09-14T07:08:44.456 INFO:teuthology.orchestra.run.smithi145.stdout:1659 read 0x130aa00 thru 0x130c150 (0x1751 bytes) 2024-09-14T07:08:44.457 INFO:teuthology.orchestra.run.smithi145.stdout:1660 punch from 0x58be800 to 0x58c6c00, (0x8400 bytes) 2024-09-14T07:08:44.457 INFO:teuthology.orchestra.run.smithi145.stdout:1663 read 0x65dae00 thru 0x65e144a (0x664b bytes) 2024-09-14T07:08:44.462 INFO:teuthology.orchestra.run.smithi145.stdout:1664 read 0xcc86800 thru 0xcc90d9d (0xa59e bytes) 2024-09-14T07:08:44.463 INFO:teuthology.orchestra.run.smithi145.stdout:1667 read 0x35a8a00 thru 0x35ae457 (0x5a58 bytes) 2024-09-14T07:08:44.464 INFO:teuthology.orchestra.run.smithi145.stdout:1669 write 0xa74400 thru 0xa788f1 (0x44f2 bytes) 2024-09-14T07:08:44.471 INFO:teuthology.orchestra.run.smithi145.stdout:1672 trunc from 0xeb3e000 to 0xc3f6600 2024-09-14T07:08:44.492 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:44.492 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:44.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:44.492 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1404 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.272156873, 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044358094000000001, 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1404 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.493 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216853+0000", 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:44.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216859+0000", 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2669999999999997e-06 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216877+0000", 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7866000000000001e-05 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216891+0000", 2024-09-14T07:08:44.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4712000000000001e-05 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260962+0000", 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070703000000003 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261081+0000", 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011867 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261210+0000", 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012960599999999999 2024-09-14T07:08:44.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1407 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.272099128000001, 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044521318999999997, 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1407 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216911+0000", 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.333e-06 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216916+0000", 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0300000000000001e-06 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216933+0000", 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6616000000000002e-05 2024-09-14T07:08:44.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216947+0000", 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3923e-05 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261260+0000", 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044313105999999998 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261323+0000", 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2688999999999996e-05 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261431+0000", 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108622 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1408 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.501 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.272043613000001, 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044764117999999999, 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1408 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216967+0000", 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4360000000000001e-06 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216987+0000", 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9689000000000001e-05 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217005+0000", 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8260000000000001e-05 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217018+0000", 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3396e-05 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261500+0000", 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044481494000000003 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261601+0000", 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101755 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261730+0000", 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128088 2024-09-14T07:08:44.505 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1561 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.267460725999999, 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044533453000000001, 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1561 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221569+0000", 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-05 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221589+0000", 2024-09-14T07:08:44.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9266000000000002e-05 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221602+0000", 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.366e-05 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265854+0000", 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044251429000000002 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265962+0000", 2024-09-14T07:08:44.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:08:44.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266082+0000", 2024-09-14T07:08:44.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011999500000000001 2024-09-14T07:08:44.510 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.511 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.511 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.511 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1562 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.267384798999998, 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044744586000000003, 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1562 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.512 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221625+0000", 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221631+0000", 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4729999999999999e-06 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221648+0000", 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7346e-05 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221663+0000", 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266141+0000", 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044478651000000001 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266249+0000", 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107371 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266369+0000", 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120258 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1563 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.267327223999999, 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044928442999999998, 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.516 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1563 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221683+0000", 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.55e-06 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221689+0000", 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1699999999999996e-06 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221706+0000", 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7286e-05 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221719+0000", 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2959000000000001e-05 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266405+0000", 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044686664000000001 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266488+0000", 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2243999999999994e-05 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266610+0000", 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012256999999999999 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.520 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1699 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.264792393, 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044262176, 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1699 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224218+0000", 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224254+0000", 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6525e-05 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224274+0000", 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0049999999999999e-05 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224289+0000", 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:08:44.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268246+0000", 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043957152999999999 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268354+0000", 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107433 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012504200000000001 2024-09-14T07:08:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1701 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.264696442999998, 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044338850999999999, 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1701 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224314+0000", 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224320+0000", 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224339+0000", 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8592000000000001e-05 2024-09-14T07:08:44.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224353+0000", 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.432e-05 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268522+0000", 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044169491999999998 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268578+0000", 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5099999999999998e-05 2024-09-14T07:08:44.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268651+0000", 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-05 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1703 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.264632761000001, 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044549686999999998, 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.529 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1703 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:44.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224378+0000", 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.573e-06 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224384+0000", 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224403+0000", 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9113000000000001e-05 2024-09-14T07:08:44.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224417+0000", 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4049000000000001e-05 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268700+0000", 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044282555000000001 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268805+0000", 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104858 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268926+0000", 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121296 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1704 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9764864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.264571419999999, 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044649934000000002, 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.533 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224439+0000", 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224902+0000", 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046329699999999999 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224924+0000", 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2059e-05 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224941+0000", 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835999999999999e-05 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268962+0000", 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044020653 2024-09-14T07:08:44.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269012+0000", 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0312000000000002e-05 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269088+0000", 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5457000000000006e-05 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1705 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.264038985999999, 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044393046999999998, 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224972+0000", 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.784e-06 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224979+0000", 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224996+0000", 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225010+0000", 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3926e-05 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269132+0000", 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044121736000000002 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269242+0000", 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010979700000000001 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269363+0000", 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121155 2024-09-14T07:08:44.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1706 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.263979824, 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044499060999999999, 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225030+0000", 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225036+0000", 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4530000000000001e-06 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225053+0000", 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6983e-05 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225066+0000", 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269399+0000", 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044332900000000001 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269449+0000", 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9960000000000003e-05 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269528+0000", 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9221999999999995e-05 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1707 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.263925253, 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044705944999999997, 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:44.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225085+0000", 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225090+0000", 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.113e-06 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225107+0000", 2024-09-14T07:08:44.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6569999999999999e-05 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225135+0000", 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8524999999999999e-05 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269571+0000", 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044435781000000001 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269676+0000", 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105374 2024-09-14T07:08:44.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269790+0000", 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113415 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1708 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.263853992000001, 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044874641, 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.550 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225156+0000", 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225162+0000", 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5960000000000003e-06 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225179+0000", 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7235999999999999e-05 2024-09-14T07:08:44.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225192+0000", 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269839+0000", 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044646439000000003 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269926+0000", 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7303000000000001e-05 2024-09-14T07:08:44.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270030+0000", 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001036 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1709 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.263797874000002, 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045094769, 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225213+0000", 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6339999999999999e-06 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225218+0000", 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3959999999999997e-06 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225236+0000", 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8053000000000001e-05 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225250+0000", 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3583e-05 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270081+0000", 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044830868000000003 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270179+0000", 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8645000000000002e-05 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270306+0000", 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012658999999999999 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1710 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.263740298999998, 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045251099000000003, 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225270+0000", 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.173e-06 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225276+0000", 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6840000000000001e-06 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225292+0000", 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6722e-05 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225306+0000", 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3456e-05 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270348+0000", 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045042722 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270431+0000", 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2182999999999996e-05 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270520+0000", 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9159000000000005e-05 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.192139826000002, 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:08:44.562 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:44.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.332664239, 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:08:44.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.571 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.332561953999999, 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:44.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:44.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:08:44.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.633510113, 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.577 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.578 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:44.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:44.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:44.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:08:44.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.578 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:44.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:08:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:08:44.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.583 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.583 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.583 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:44.584 INFO:teuthology.orchestra.run.smithi145.stdout:1673 write 0xaf7ce00 thru 0xaf84a86 (0x7c87 bytes) 2024-09-14T07:08:44.584 INFO:teuthology.orchestra.run.smithi145.stdout:1675 trunc from 0xc3f6600 to 0x3a35e00 2024-09-14T07:08:44.585 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:44.585 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:44.581+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:44.597 INFO:teuthology.orchestra.run.smithi145.stdout:1676 trunc from 0x3a35e00 to 0x1f3a000 2024-09-14T07:08:44.685 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:44.681+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:44.685 INFO:teuthology.orchestra.run.smithi145.stdout:1681 write 0xd173800 thru 0xd17b726 (0x7f27 bytes) 2024-09-14T07:08:44.690 INFO:teuthology.orchestra.run.smithi145.stdout:1682 write 0x2fe9000 thru 0x2ff8d24 (0xfd25 bytes) 2024-09-14T07:08:44.692 INFO:teuthology.orchestra.run.smithi145.stdout:1683 write 0x452a00 thru 0x45a9b6 (0x7fb7 bytes) 2024-09-14T07:08:44.710 INFO:teuthology.orchestra.run.smithi145.stdout:1687 trunc from 0xd17b800 to 0x5d3ee00 2024-09-14T07:08:44.718 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:44.718 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:44.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4867.0:3251 3.16 3:6f1242db:::rbd_data.1161ee0db696.0000000000000002:head [write 3342336~36352 in=36352b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:44.712714+0000", 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0057981819999999998, 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058205419999999997, 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "waiting for sub ops", 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4867", 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3251 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:44.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.712714+0000", 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.712714+0000", 2024-09-14T07:08:44.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:44.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:44.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.712717+0000", 2024-09-14T07:08:44.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7130000000000001e-06 2024-09-14T07:08:44.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:44.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.712790+0000", 2024-09-14T07:08:44.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3025e-05 2024-09-14T07:08:44.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.712820+0000", 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0216e-05 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.712841+0000", 2024-09-14T07:08:44.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0557e-05 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.712929+0000", 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8453000000000002e-05 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.713002+0000", 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3089000000000004e-05 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.713104+0000", 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101584 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:44.716071+0000", 2024-09-14T07:08:44.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002967707 2024-09-14T07:08:44.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.726 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:44.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:44.726 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:08:44.726 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:08:44.726 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:44.734 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:44.753 INFO:teuthology.orchestra.run.smithi145.stdout:1688 read 0x234e000 thru 0x235d0ce (0xf0cf bytes) 2024-09-14T07:08:44.801 INFO:teuthology.orchestra.run.smithi145.stdout:1692 write 0xd5f6a00 thru 0xd5fcbc4 (0x61c5 bytes) 2024-09-14T07:08:44.803 INFO:teuthology.orchestra.run.smithi145.stdout:1693 punch from 0x9e7a400 to 0x9e7bc00, (0x1800 bytes) 2024-09-14T07:08:44.804 INFO:teuthology.orchestra.run.smithi145.stdout:1694 read 0x53d3c00 thru 0x53d74fe (0x38ff bytes) 2024-09-14T07:08:44.806 INFO:teuthology.orchestra.run.smithi145.stdout:1696 write 0x27ed200 thru 0x27f024d (0x304e bytes) 2024-09-14T07:08:44.807 INFO:teuthology.orchestra.run.smithi145.stdout:1697 read 0xb2ef600 thru 0xb2f95d2 (0x9fd3 bytes) 2024-09-14T07:08:44.809 INFO:teuthology.orchestra.run.smithi145.stdout:1698 read 0xa088400 thru 0xa095b34 (0xd735 bytes) 2024-09-14T07:08:44.810 INFO:teuthology.orchestra.run.smithi145.stdout:1699 read 0x90eee00 thru 0x90efa8f (0xc90 bytes) 2024-09-14T07:08:44.811 INFO:teuthology.orchestra.run.smithi145.stdout:1700 read 0x92bf600 thru 0x92c93da (0x9ddb bytes) 2024-09-14T07:08:44.812 INFO:teuthology.orchestra.run.smithi145.stdout:1701 write 0x1f6600 thru 0x204a67 (0xe468 bytes) 2024-09-14T07:08:44.813 INFO:teuthology.orchestra.run.smithi145.stdout:1707 punch from 0xc5ba400 to 0xc5c8e00, (0xea00 bytes) 2024-09-14T07:08:44.814 INFO:teuthology.orchestra.run.smithi145.stdout:1710 read 0x73db400 thru 0x73e4f39 (0x9b3a bytes) 2024-09-14T07:08:44.823 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:44.821+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:44.828 INFO:teuthology.orchestra.run.smithi145.stdout:1711 trunc from 0xd5fcc00 to 0xa87ac00 2024-09-14T07:08:44.860 INFO:teuthology.orchestra.run.smithi145.stdout:1713 punch from 0x55f9000 to 0x55fa800, (0x1800 bytes) 2024-09-14T07:08:44.862 INFO:teuthology.orchestra.run.smithi145.stdout:1715 trunc from 0xa87ac00 to 0xabb8c00 2024-09-14T07:08:44.886 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:44.881+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:44.887 INFO:teuthology.orchestra.run.smithi145.stdout:1716 read 0x9028e00 thru 0x902a0c8 (0x12c9 bytes) 2024-09-14T07:08:44.908 INFO:teuthology.orchestra.run.smithi145.stdout:1718 trunc from 0xabb8c00 to 0x3554a00 2024-09-14T07:08:44.914 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:44.915 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:44.932 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:44.945 INFO:teuthology.orchestra.run.smithi145.stdout:truncating to largest ever: 0xea66200 2024-09-14T07:08:44.973 INFO:teuthology.orchestra.run.smithi145.stdout:1720 trunc from 0x3554a00 to 0xea66200 2024-09-14T07:08:44.987 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:44.985+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:45.037 INFO:teuthology.orchestra.run.smithi145.stdout:1722 trunc from 0xea66200 to 0x1149c00 2024-09-14T07:08:45.087 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:45.085+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:45.098 INFO:teuthology.orchestra.run.smithi145.stdout:1723 read 0xa81a00 thru 0xa84c71 (0x3272 bytes) 2024-09-14T07:08:45.102 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:45.102 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 52.269495069999998, 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:45.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:45.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 52.268720721999998, 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:45.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3038 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:45.113 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970498547999998, 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046439165999999997, 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3038 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129535+0000", 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3230000000000001e-06 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129541+0000", 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.054e-06 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129559+0000", 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8329e-05 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129573+0000", 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175474+0000", 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045901019000000001 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175554+0000", 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0661000000000006e-05 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175973+0000", 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041842300000000002 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3039 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.117 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970440868000001, 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046967035999999997, 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3039 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:45.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129592+0000", 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.206e-06 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129598+0000", 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2340000000000002e-06 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129616+0000", 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8003e-05 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129629+0000", 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2913e-05 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176018+0000", 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046389224999999999 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176108+0000", 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176558+0000", 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044997100000000002 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3040 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.121 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:45.122 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970351665999999, 2024-09-14T07:08:45.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047451337000000003, 2024-09-14T07:08:45.122 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.122 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.122 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.123 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3040 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:45.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129682+0000", 2024-09-14T07:08:45.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.607e-06 2024-09-14T07:08:45.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129688+0000", 2024-09-14T07:08:45.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129708+0000", 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0047e-05 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129723+0000", 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.502e-05 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176597+0000", 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046873259 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176684+0000", 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7349999999999995e-05 2024-09-14T07:08:45.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177132+0000", 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044784099999999998 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.129 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3041 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970288363000002, 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047622767000000003, 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3041 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129745+0000", 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129751+0000", 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129773+0000", 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782999999999999e-05 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129788+0000", 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4606999999999999e-05 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177167+0000", 2024-09-14T07:08:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047378941000000001 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177253+0000", 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5704999999999993e-05 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177367+0000", 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113861 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3042 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970224924, 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048188976000000001, 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3042 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129808+0000", 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129814+0000", 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7699999999999998e-06 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129832+0000", 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.751e-05 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129844+0000", 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2797e-05 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177436+0000", 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047591574999999997 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177545+0000", 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109446 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177996+0000", 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045071800000000002 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3043 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970168470000001, 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048376965000000001, 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3043 2024-09-14T07:08:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129865+0000", 2024-09-14T07:08:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129870+0000", 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129888+0000", 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.755e-05 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129902+0000", 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178049+0000", 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048146648 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178131+0000", 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2619000000000003e-05 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178241+0000", 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109262 2024-09-14T07:08:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3044 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970096390999998, 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048575449, 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3044 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129937+0000", 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1930000000000001e-06 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129943+0000", 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129961+0000", 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8034e-05 2024-09-14T07:08:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129975+0000", 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3696000000000001e-05 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178293+0000", 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048318051000000001 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178392+0000", 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9038999999999998e-05 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178511+0000", 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119493 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3045 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.970037134999998, 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049079214000000003, 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3045 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129996+0000", 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1170000000000001e-06 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130002+0000", 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-14T07:08:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130024+0000", 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1763000000000002e-05 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130059+0000", 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.536e-05 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178555+0000", 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048495599 2024-09-14T07:08:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178636+0000", 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1112000000000005e-05 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179074+0000", 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043820000000000003 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3046 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.969950459, 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049550174000000002, 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3046 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130083+0000", 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130089+0000", 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-14T07:08:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130108+0000", 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9046999999999999e-05 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130122+0000", 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4137e-05 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179126+0000", 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049003927000000003 2024-09-14T07:08:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179221+0000", 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4717999999999997e-05 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179632+0000", 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041092199999999998 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3047 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.969889762000001, 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050053944000000003, 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3047 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130144+0000", 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3030000000000001e-06 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130149+0000", 2024-09-14T07:08:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6230000000000002e-06 2024-09-14T07:08:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130169+0000", 2024-09-14T07:08:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9386999999999999e-05 2024-09-14T07:08:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130182+0000", 2024-09-14T07:08:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3706000000000001e-05 2024-09-14T07:08:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179677+0000", 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049494109000000001 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179763+0000", 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6473999999999999e-05 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180196+0000", 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043334200000000003 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3048 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.969829842999999, 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050582655999999997, 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3048 2024-09-14T07:08:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130204+0000", 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.266e-06 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130219+0000", 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5549999999999999e-05 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130235+0000", 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6030000000000001e-05 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130249+0000", 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3977e-05 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180264+0000", 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050014851999999999 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180359+0000", 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5400000000000001e-05 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180785+0000", 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042558100000000002 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3049 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.969762376999999, 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.051156504999999998, 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3049 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130271+0000", 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130277+0000", 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6370000000000004e-06 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130297+0000", 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.997e-05 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130311+0000", 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4358999999999999e-05 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180833+0000", 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050521863 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180936+0000", 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010296799999999999 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.181426+0000", 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049044799999999999 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3149 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.954575801000001, 2024-09-14T07:08:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046930569999999998, 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3149 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145458+0000", 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145463+0000", 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6169999999999998e-06 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145486+0000", 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2626e-05 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145500+0000", 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4243000000000001e-05 2024-09-14T07:08:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191669+0000", 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046169090000000003 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191811+0000", 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014187100000000001 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192387+0000", 2024-09-14T07:08:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057567299999999996 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3150 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.954509440999999, 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047680769999999997, 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3150 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145524+0000", 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.39e-06 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145647+0000", 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123242 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145673+0000", 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145711+0000", 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8745999999999998e-05 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192444+0000", 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046732462000000002 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192602+0000", 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015825400000000001 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193203+0000", 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00060150600000000002 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3172 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.954123571, 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048262865000000002, 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3172 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145911+0000", 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145920+0000", 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2969999999999992e-06 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145947+0000", 2024-09-14T07:08:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7149999999999999e-05 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145965+0000", 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8295999999999999e-05 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193303+0000", 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047338247 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193492+0000", 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018843500000000001 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194171+0000", 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067951700000000003 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3173 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.954038215000001, 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048937952999999999, 2024-09-14T07:08:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3173 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145995+0000", 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146021+0000", 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5684e-05 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146044+0000", 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2503000000000001e-05 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146059+0000", 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5043e-05 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194258+0000", 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04819975 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194372+0000", 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113373 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194932+0000", 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056004399999999999 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:45.186 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.257537723999999, 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:08:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.960609517999998, 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:08:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:08:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:08:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:08:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:08:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:45.197 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:45.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:45.199 INFO:teuthology.orchestra.run.smithi145.stdout:1727 write 0x31b400 thru 0x325f20 (0xab21 bytes) 2024-09-14T07:08:45.200 INFO:teuthology.orchestra.run.smithi145.stdout:1728 read 0xa4aa00 thru 0xa54ff0 (0xa5f1 bytes) 2024-09-14T07:08:45.200 INFO:teuthology.orchestra.run.smithi145.stdout:1729 read 0xd77e00 thru 0xd8394b (0xbb4c bytes) 2024-09-14T07:08:45.200 INFO:teuthology.orchestra.run.smithi145.stdout:1732 trunc from 0x1149c00 to 0xe45ea00 2024-09-14T07:08:45.200 INFO:teuthology.orchestra.run.smithi145.stdout:1733 trunc from 0xe45ea00 to 0x8147400 2024-09-14T07:08:45.200 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:45.201 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:45.197+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:45.254 INFO:teuthology.orchestra.run.smithi145.stdout:1734 write 0xd874400 thru 0xd87ecfc (0xa8fd bytes) 2024-09-14T07:08:45.257 INFO:teuthology.orchestra.run.smithi145.stdout:1735 read 0x823bc00 thru 0x8246b58 (0xaf59 bytes) 2024-09-14T07:08:45.281 INFO:teuthology.orchestra.run.smithi145.stdout:1736 trunc from 0xd87ee00 to 0x3957800 2024-09-14T07:08:45.300 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:45.297+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:45.321 INFO:teuthology.orchestra.run.smithi145.stdout:1742 punch from 0x9f8600 to 0xa00200, (0x7c00 bytes) 2024-09-14T07:08:45.324 INFO:teuthology.orchestra.run.smithi145.stdout:1744 read 0x137800 thru 0x141176 (0x9977 bytes) 2024-09-14T07:08:45.325 INFO:teuthology.orchestra.run.smithi145.stdout:1745 clone 13 order 25 su 262144 sc 6 2024-09-14T07:08:45.339 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:45.339 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:45.339 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:45.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:45.361 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:45.361 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:45.378 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:45.379 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:45.395 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:45.395 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:45.400 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:45.397+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:45.416 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:45.437 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:45.437 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:45.458 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:45.458 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:45.472 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:45.472 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:45.487 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:45.532 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:45.529+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:45.602 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:45.597+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:45.648 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:45.649 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:45.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:45.703 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:45.701+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:45.804 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:45.801+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:45.833 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:45.833 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:45.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:45.905 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:45.901+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:45.974 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:45.974 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:45.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:46.006 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:46.005+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:46.106 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:46.101+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:46.163 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:46.163 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:46.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:46.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:08:46.206 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:46.201+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:46.308 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:46.305+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:46.315 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:46.330 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:46.409 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:46.405+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:46.451 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:46.451 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:46.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:46.510 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:46.505+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:46.610 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:46.605+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:46.625 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:46.625 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:46.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:46.711 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:46.709+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:46.775 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T07:08:46.776 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:46.777 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:46.792 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:46.835 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:46.833+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:46.912 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:46.909+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:46.920 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:46.920 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:46.935 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:47.019 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:47.017+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:47.104 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:47.104 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:47.114 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:47.109+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:47.119 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:08:47.215 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:47.209+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:47.285 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:47.285 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:47.296 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 5, 2, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:47.296 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:47.296 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 3, 5, 2, 4] 2024-09-14T07:08:47.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-14T07:08:47.299 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:47.344 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:47.341+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:47.416 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:47.413+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:47.463 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:47.464 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:47.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:47.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:47.520 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:47.517+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:47.617 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:47.618 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:47.613+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:47.634 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:47.636 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:47.636 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:47.640 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:08:47.637+0000 7f1e6b537640 -1 mon.a@0(leader).osd e67 definitely_dead 0 2024-09-14T07:08:47.650 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:47.651 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:47.662 INFO:teuthology.orchestra.run.smithi145.stdout:1747 punch from 0xf20600 to 0xf2e000, (0xda00 bytes) 2024-09-14T07:08:47.668 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:47.668 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:47.689 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:47.689 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:47.706 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:47.719 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:47.717+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:47.742 INFO:teuthology.orchestra.run.smithi145.stdout:1749 trunc from 0x3957800 to 0xd8c7400 2024-09-14T07:08:47.758 INFO:teuthology.orchestra.run.smithi145.stdout:1751 trunc from 0xd8c7400 to 0xd50d600 2024-09-14T07:08:47.775 INFO:teuthology.orchestra.run.smithi145.stdout:1752 read 0xf3a400 thru 0xf400f0 (0x5cf1 bytes) 2024-09-14T07:08:47.778 INFO:teuthology.orchestra.run.smithi145.stdout:1754 read 0x3fa2c00 thru 0x3fa5ea0 (0x32a1 bytes) 2024-09-14T07:08:47.778 INFO:teuthology.orchestra.run.smithi145.stdout:1755 write 0x7b68600 thru 0x7b6dd0d (0x570e bytes) 2024-09-14T07:08:47.791 INFO:teuthology.orchestra.run.smithi145.stdout:1756 trunc from 0xd50d600 to 0x6050e00 2024-09-14T07:08:47.820 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:47.817+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:527 3.16 3.db4248f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.826013+0000", 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00081079199999999996, 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00086225799999999997, 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.831 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 527 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826013+0000", 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826013+0000", 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826014+0000", 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8000000000000004e-07 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826018+0000", 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-06 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826027+0000", 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6330000000000008e-06 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826034+0000", 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-06 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:528 3.16 3.db4248f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.826049+0000", 2024-09-14T07:08:47.834 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00077484199999999998, 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00099846200000000009, 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 528 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826049+0000", 2024-09-14T07:08:47.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826049+0000", 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826050+0000", 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7300000000000004e-07 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826055+0000", 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8570000000000003e-06 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826071+0000", 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6269e-05 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826081+0000", 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6630000000000005e-06 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.837 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:529 3.16 3.db4248f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.826099+0000", 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00072519999999999995, 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011024400000000001, 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 529 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826099+0000", 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826099+0000", 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826100+0000", 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.263e-06 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826105+0000", 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2970000000000003e-06 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826123+0000", 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7428999999999999e-05 2024-09-14T07:08:47.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826136+0000", 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2999999999999999e-05 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:530 3.16 3.db4248f6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.826153+0000", 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00067083400000000004, 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001194535, 2024-09-14T07:08:47.841 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826153+0000", 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826153+0000", 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826154+0000", 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.093e-06 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826159+0000", 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1830000000000002e-06 2024-09-14T07:08:47.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826176+0000", 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6943e-05 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826189+0000", 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2490000000000001e-05 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:575 3.1d 3:be48dfa2:::rbd_data.1161ee0db696.0000000000000007:f [sparse-read 3473408~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.826643+0000", 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00018044, 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084770899999999998, 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 575 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.845 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826643+0000", 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826643+0000", 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826645+0000", 2024-09-14T07:08:47.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826651+0000", 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826670+0000", 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9392999999999999e-05 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.826686+0000", 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5393e-05 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827351+0000", 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066593999999999996 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827413+0000", 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1729999999999993e-05 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827491+0000", 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7925999999999999e-05 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:608 3.1c 3:3c0a7c4a:::rbd_data.1161ee0db696.000000000000000b:f [sparse-read 196608~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827204+0000", 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9996204, 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052231000000000001, 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.849 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 608 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827204+0000", 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827204+0000", 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827204+0000", 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0699999999999996e-07 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827207+0000", 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9359999999999999e-06 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827215+0000", 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9880000000000004e-06 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827221+0000", 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9739999999999999e-06 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827618+0000", 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00039666099999999998 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827659+0000", 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1114e-05 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827726+0000", 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6929999999999998e-05 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:609 3.1c 3:3c0a7c4a:::rbd_data.1161ee0db696.000000000000000b:f [sparse-read 524288~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827231+0000", 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995928, 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00068665100000000004, 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 609 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827231+0000", 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827231+0000", 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827231+0000", 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:08:47.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827234+0000", 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.824e-06 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827241+0000", 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4849999999999997e-06 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827246+0000", 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7440000000000004e-06 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827754+0000", 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050785800000000003 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827828+0000", 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3312000000000006e-05 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827918+0000", 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9887000000000002e-05 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:610 3.1c 3:3c0a7c4a:::rbd_data.1161ee0db696.000000000000000b:f [sparse-read 720896~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827255+0000", 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995689, 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00090317800000000003, 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 610 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827255+0000", 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827255+0000", 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827255+0000", 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0200000000000001e-07 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827258+0000", 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2220000000000001e-06 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827264+0000", 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2319999999999998e-06 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827269+0000", 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2619999999999998e-06 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827980+0000", 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000711238 2024-09-14T07:08:47.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828149+0000", 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016882600000000001 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:611 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827277+0000", 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995465, 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010223859999999999, 2024-09-14T07:08:47.862 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 611 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827277+0000", 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827277+0000", 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827278+0000", 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8400000000000005e-07 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827280+0000", 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:08:47.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827287+0000", 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5209999999999999e-06 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827292+0000", 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2100000000000001e-06 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:612 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827309+0000", 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995146, 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001143972, 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 612 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827309+0000", 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827309+0000", 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827310+0000", 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.68e-07 2024-09-14T07:08:47.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827313+0000", 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827320+0000", 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5920000000000003e-06 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827326+0000", 2024-09-14T07:08:47.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8359999999999997e-06 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:613 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827336+0000", 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9994879, 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012660869999999999, 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 613 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827336+0000", 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827336+0000", 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827337+0000", 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827339+0000", 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4140000000000001e-06 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827346+0000", 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9959999999999996e-06 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827351+0000", 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2959999999999998e-06 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:614 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827363+0000", 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9994612, 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013851, 2024-09-14T07:08:47.872 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 614 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827363+0000", 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827363+0000", 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827364+0000", 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0399999999999995e-07 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827366+0000", 2024-09-14T07:08:47.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3259999999999998e-06 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827373+0000", 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8870000000000003e-06 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827378+0000", 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4940000000000003e-06 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:615 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827386+0000", 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9994378, 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015177579999999999, 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 615 2024-09-14T07:08:47.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827386+0000", 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827386+0000", 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827387+0000", 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827389+0000", 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.035e-06 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827395+0000", 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3389999999999998e-06 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827400+0000", 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0860000000000001e-06 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:616 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827408+0000", 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9994164, 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016481169999999999, 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.879 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 616 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827408+0000", 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827408+0000", 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827408+0000", 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827410+0000", 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.007e-06 2024-09-14T07:08:47.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827416+0000", 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3060000000000003e-06 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827421+0000", 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:617 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827429+0000", 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9993954, 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017773190000000001, 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 617 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827429+0000", 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827429+0000", 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827429+0000", 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5600000000000001e-07 2024-09-14T07:08:47.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827431+0000", 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2649999999999999e-06 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827438+0000", 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.387e-06 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827443+0000", 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:618 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827453+0000", 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9993711, 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001906228, 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 618 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827453+0000", 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827453+0000", 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827453+0000", 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827456+0000", 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3379999999999999e-06 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827462+0000", 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6969999999999996e-06 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827469+0000", 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0349999999999998e-06 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:619 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827476+0000", 2024-09-14T07:08:47.889 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9993477, 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020425700000000001, 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 619 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827476+0000", 2024-09-14T07:08:47.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827476+0000", 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827477+0000", 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6499999999999999e-07 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827479+0000", 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.176e-06 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827485+0000", 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4740000000000002e-06 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827490+0000", 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0989999999999998e-06 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:620 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827497+0000", 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9993267, 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002167329, 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 620 2024-09-14T07:08:47.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827497+0000", 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827497+0000", 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827498+0000", 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.03e-07 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827500+0000", 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0710000000000002e-06 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827506+0000", 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9619999999999998e-06 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827510+0000", 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:621 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827528+0000", 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9992962, 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002215491, 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 621 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827528+0000", 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827528+0000", 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827528+0000", 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1999999999999999e-07 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827531+0000", 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7010000000000001e-06 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827538+0000", 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3039999999999999e-06 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827543+0000", 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1599999999999997e-06 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:622 3.1c 3.523e503c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.899 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827552+0000", 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9992719, 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002263162, 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 622 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827552+0000", 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827552+0000", 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827553+0000", 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.44e-07 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827555+0000", 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827563+0000", 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3089999999999996e-06 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827568+0000", 2024-09-14T07:08:47.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0289999999999996e-06 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:631 3.e 3:77ea976f:::rbd_data.1161ee0db696.000000000000000d:f [sparse-read 196608~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827874+0000", 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.99895, 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020233149999999999, 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.903 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 631 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827874+0000", 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827874+0000", 2024-09-14T07:08:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827875+0000", 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0999999999999997e-07 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827879+0000", 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4700000000000004e-06 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827888+0000", 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7439999999999996e-06 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827895+0000", 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1219999999999997e-06 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.829550+0000", 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001655445 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:47.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.829777+0000", 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000226336 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:632 3.e 3.f6e957ee (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827908+0000", 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9989157, 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020746200000000001, 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 632 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827908+0000", 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827908+0000", 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827909+0000", 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5700000000000002e-07 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827911+0000", 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6280000000000001e-06 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827918+0000", 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5230000000000001e-06 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827923+0000", 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2020000000000003e-06 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:633 3.e 3.f6e957ee (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.827967+0000", 2024-09-14T07:08:47.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.998857, 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00208785, 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 633 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827967+0000", 2024-09-14T07:08:47.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827967+0000", 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827968+0000", 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827971+0000", 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4450000000000001e-06 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827979+0000", 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3130000000000001e-06 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.827985+0000", 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8139999999999997e-06 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.913 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:634 3.e 3.f6e957ee (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.828004+0000", 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9988198, 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021219490000000001, 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 634 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828004+0000", 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828004+0000", 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828005+0000", 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6700000000000003e-07 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828007+0000", 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7880000000000001e-06 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828011+0000", 2024-09-14T07:08:47.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4270000000000001e-06 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828015+0000", 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8639999999999998e-06 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:635 3.e 3.f6e957ee (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e67)", 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:47.828023+0000", 2024-09-14T07:08:47.917 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9988012, 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021746090000000001, 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 635 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:47.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828023+0000", 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828023+0000", 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828023+0000", 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.75e-07 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828025+0000", 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3130000000000001e-06 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828031+0000", 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8250000000000001e-06 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:47.828036+0000", 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3089999999999997e-06 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.920 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:47.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:47.921 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:08:47.921 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 25 2024-09-14T07:08:47.921 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:47.922 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:47.923 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:47.917+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:47.985 INFO:teuthology.orchestra.run.smithi145.stdout:1758 read 0x344d400 thru 0x344d7f5 (0x3f6 bytes) 2024-09-14T07:08:48.009 INFO:teuthology.orchestra.run.smithi145.stdout:1759 write 0x6fd2600 thru 0x6fdfe26 (0xd827 bytes) 2024-09-14T07:08:48.011 INFO:teuthology.orchestra.run.smithi145.stdout:1761 punch from 0x3514400 to 0x351b800, (0x7400 bytes) 2024-09-14T07:08:48.022 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:48.017+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:48.038 INFO:teuthology.orchestra.run.smithi145.stdout:1767 write 0xa091000 thru 0xa09637b (0x537c bytes) 2024-09-14T07:08:48.041 INFO:teuthology.orchestra.run.smithi145.stdout:1770 read 0x93d9e00 thru 0x93e6b44 (0xcd45 bytes) 2024-09-14T07:08:48.042 INFO:teuthology.orchestra.run.smithi145.stdout:1773 write 0x7df7200 thru 0x7dfe2f0 (0x70f1 bytes) 2024-09-14T07:08:48.044 INFO:teuthology.orchestra.run.smithi145.stdout:1774 read 0x62de000 thru 0x62dea7c (0xa7d bytes) 2024-09-14T07:08:48.054 INFO:teuthology.orchestra.run.smithi145.stdout:1777 trunc from 0xa096400 to 0xd6c3600 2024-09-14T07:08:48.063 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:48.063 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:48.063 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:48.063 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:48.063 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:48.080 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:48.084 INFO:teuthology.orchestra.run.smithi145.stdout:1780 write 0xbaaf800 thru 0xbabe047 (0xe848 bytes) 2024-09-14T07:08:48.086 INFO:teuthology.orchestra.run.smithi145.stdout:1783 read 0xba47000 thru 0xba514e3 (0xa4e4 bytes) 2024-09-14T07:08:48.087 INFO:teuthology.orchestra.run.smithi145.stdout:1785 write 0x52cfc00 thru 0x52d021a (0x61b bytes) 2024-09-14T07:08:48.092 INFO:teuthology.orchestra.run.smithi145.stdout:1788 trunc from 0xd6c3600 to 0xc840200 2024-09-14T07:08:48.124 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:48.121+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:48.146 INFO:teuthology.orchestra.run.smithi145.stdout:1792 trunc from 0xc840200 to 0x2f79a00 2024-09-14T07:08:48.177 INFO:teuthology.orchestra.run.smithi145.stdout:1793 read 0x2502200 thru 0x2502f89 (0xd8a bytes) 2024-09-14T07:08:48.187 INFO:teuthology.orchestra.run.smithi145.stdout:1795 trunc from 0x2f79a00 to 0x1f6da00 2024-09-14T07:08:48.202 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.0. 2024-09-14T07:08:48.225 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:48.221+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.424392284, 2024-09-14T07:08:48.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:48.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:48.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:48.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:48.263 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.423617935999999, 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:48.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:48.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3038 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.125395761999997, 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046439165999999997, 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3038 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129535+0000", 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3230000000000001e-06 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129541+0000", 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.054e-06 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129559+0000", 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8329e-05 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129573+0000", 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175474+0000", 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045901019000000001 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175554+0000", 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0661000000000006e-05 2024-09-14T07:08:48.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175973+0000", 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041842300000000002 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3039 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.125338081999999, 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046967035999999997, 2024-09-14T07:08:48.274 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3039 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129592+0000", 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.206e-06 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129598+0000", 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2340000000000002e-06 2024-09-14T07:08:48.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129616+0000", 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8003e-05 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129629+0000", 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2913e-05 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176018+0000", 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046389224999999999 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176108+0000", 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176558+0000", 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044997100000000002 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3040 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.125248880000001, 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047451337000000003, 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3040 2024-09-14T07:08:48.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129682+0000", 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.607e-06 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129688+0000", 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129708+0000", 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0047e-05 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129723+0000", 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.502e-05 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176597+0000", 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046873259 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176684+0000", 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7349999999999995e-05 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177132+0000", 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044784099999999998 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.284 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3041 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.284 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:48.284 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.125185577000003, 2024-09-14T07:08:48.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047622767000000003, 2024-09-14T07:08:48.285 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.285 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.285 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.286 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3041 2024-09-14T07:08:48.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.286 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129745+0000", 2024-09-14T07:08:48.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-14T07:08:48.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129751+0000", 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129773+0000", 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782999999999999e-05 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129788+0000", 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4606999999999999e-05 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177167+0000", 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047378941000000001 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177253+0000", 2024-09-14T07:08:48.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5704999999999993e-05 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177367+0000", 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113861 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3042 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.125122138000002, 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048188976000000001, 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3042 2024-09-14T07:08:48.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129808+0000", 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129814+0000", 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7699999999999998e-06 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129832+0000", 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.751e-05 2024-09-14T07:08:48.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129844+0000", 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2797e-05 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177436+0000", 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047591574999999997 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177545+0000", 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109446 2024-09-14T07:08:48.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177996+0000", 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045071800000000002 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3043 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.125065683999999, 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048376965000000001, 2024-09-14T07:08:48.296 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3043 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129865+0000", 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129870+0000", 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:48.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129888+0000", 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.755e-05 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129902+0000", 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178049+0000", 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048146648 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178131+0000", 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2619000000000003e-05 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178241+0000", 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109262 2024-09-14T07:08:48.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3044 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.124993605, 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048575449, 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3044 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129937+0000", 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1930000000000001e-06 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129943+0000", 2024-09-14T07:08:48.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129961+0000", 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8034e-05 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129975+0000", 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3696000000000001e-05 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178293+0000", 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048318051000000001 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178392+0000", 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9038999999999998e-05 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178511+0000", 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119493 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3045 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.124934349, 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049079214000000003, 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3045 2024-09-14T07:08:48.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129996+0000", 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1170000000000001e-06 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130002+0000", 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130024+0000", 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1763000000000002e-05 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130059+0000", 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.536e-05 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178555+0000", 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048495599 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178636+0000", 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1112000000000005e-05 2024-09-14T07:08:48.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179074+0000", 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043820000000000003 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3046 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.124847672999998, 2024-09-14T07:08:48.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049550174000000002, 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3046 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130083+0000", 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130089+0000", 2024-09-14T07:08:48.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130108+0000", 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9046999999999999e-05 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130122+0000", 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4137e-05 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179126+0000", 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049003927000000003 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179221+0000", 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4717999999999997e-05 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179632+0000", 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041092199999999998 2024-09-14T07:08:48.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3047 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.124786976000003, 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050053944000000003, 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3047 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130144+0000", 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3030000000000001e-06 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130149+0000", 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6230000000000002e-06 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130169+0000", 2024-09-14T07:08:48.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9386999999999999e-05 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130182+0000", 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3706000000000001e-05 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179677+0000", 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049494109000000001 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179763+0000", 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6473999999999999e-05 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180196+0000", 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043334200000000003 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3048 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.124727057000001, 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050582655999999997, 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3048 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.387 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:48.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130204+0000", 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.266e-06 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130219+0000", 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5549999999999999e-05 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130235+0000", 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6030000000000001e-05 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130249+0000", 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3977e-05 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180264+0000", 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050014851999999999 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180359+0000", 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5400000000000001e-05 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180785+0000", 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042558100000000002 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3049 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.124659590999997, 2024-09-14T07:08:48.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.051156504999999998, 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3049 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130271+0000", 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130277+0000", 2024-09-14T07:08:48.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6370000000000004e-06 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130297+0000", 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.997e-05 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130311+0000", 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4358999999999999e-05 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180833+0000", 2024-09-14T07:08:48.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050521863 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180936+0000", 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010296799999999999 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.181426+0000", 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049044799999999999 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.396 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3149 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.109473014999999, 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046930569999999998, 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.397 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3149 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145458+0000", 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145463+0000", 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6169999999999998e-06 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145486+0000", 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2626e-05 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145500+0000", 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4243000000000001e-05 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191669+0000", 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046169090000000003 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191811+0000", 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014187100000000001 2024-09-14T07:08:48.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192387+0000", 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057567299999999996 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3150 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.109406655000001, 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047680769999999997, 2024-09-14T07:08:48.401 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.402 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.402 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.404 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3150 2024-09-14T07:08:48.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.404 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145524+0000", 2024-09-14T07:08:48.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.39e-06 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145647+0000", 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123242 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145673+0000", 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145711+0000", 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8745999999999998e-05 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192444+0000", 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046732462000000002 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192602+0000", 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015825400000000001 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193203+0000", 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00060150600000000002 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3172 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.408 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.109020784999998, 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048262865000000002, 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3172 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145911+0000", 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145920+0000", 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2969999999999992e-06 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145947+0000", 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7149999999999999e-05 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145965+0000", 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8295999999999999e-05 2024-09-14T07:08:48.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193303+0000", 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047338247 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193492+0000", 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018843500000000001 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194171+0000", 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067951700000000003 2024-09-14T07:08:48.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3173 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.108935428999999, 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048937952999999999, 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:48.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3173 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:48.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145995+0000", 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146021+0000", 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5684e-05 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146044+0000", 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2503000000000001e-05 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146059+0000", 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5043e-05 2024-09-14T07:08:48.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194258+0000", 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04819975 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194372+0000", 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113373 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194932+0000", 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056004399999999999 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.418 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.412434938000001, 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:08:48.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:08:48.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.115506732, 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:48.425 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:48.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:48.431 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:48.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:48.434 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:48.429+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:48.534 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:48.529+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:48.568 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:48.569 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:48.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:48.635 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:48.633+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:48.712 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:48.712 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:48.712 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:48.712 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:48.712 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:48.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:48.736 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:48.733+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:48.837 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:48.833+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:48.855 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.566096157, 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:08:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.536588525999999, 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.527941167000002, 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:08:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:08:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.861848826999999, 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:08:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.695119601000002, 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:08:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.513188087, 2024-09-14T07:08:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:08:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:08:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.762118593, 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:08:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:08:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:08:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:08:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:08:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:08:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4732.0:183 3.1b 3:d96d4dd6:::rbd_data.1161bf39a192.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 6709248~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.912248553000001, 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030801473999999999, 2024-09-14T07:08:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4732", 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/303097868", 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 183 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942590+0000", 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-09-14T07:08:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944540+0000", 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001949824 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944564+0000", 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4199e-05 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944574+0000", 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944644+0000", 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0240999999999994e-05 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944798+0000", 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153432 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.945082+0000", 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028403899999999998 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.959401+0000", 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014319159999999999 2024-09-14T07:08:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973083+0000", 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013682016 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973261+0000", 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017752999999999999 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973390+0000", 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012949500000000001 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:538 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9205629240000004, 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020569724000000001, 2024-09-14T07:08:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 538 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934275+0000", 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934281+0000", 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934299+0000", 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8470000000000001e-05 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934313+0000", 2024-09-14T07:08:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3196e-05 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954450+0000", 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020137616000000001 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954629+0000", 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017857499999999999 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954844+0000", 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021515099999999999 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:539 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9204411740000005, 2024-09-14T07:08:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02104733, 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 539 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:08:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934398+0000", 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.813e-06 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934404+0000", 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7460000000000003e-06 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934425+0000", 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.033e-05 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934440+0000", 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5019e-05 2024-09-14T07:08:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954916+0000", 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020476173 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955086+0000", 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169655 2024-09-14T07:08:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955443+0000", 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035759399999999998 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:540 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:08:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9203735789999996, 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021617992999999999, 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 540 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934465+0000", 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4130000000000001e-06 2024-09-14T07:08:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934471+0000", 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7259999999999999e-06 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934489+0000", 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8263000000000001e-05 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934506+0000", 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955539+0000", 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021033046999999999 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955813+0000", 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00027412200000000002 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956082+0000", 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000268829 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:541 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9203086410000001, 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022127142999999998, 2024-09-14T07:08:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 541 2024-09-14T07:08:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934530+0000", 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.037e-06 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934535+0000", 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8259999999999998e-06 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934554+0000", 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934567+0000", 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3573e-05 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956183+0000", 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021615059999999998 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956395+0000", 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021209399999999999 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956656+0000", 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026098000000000001 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:542 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9202496499999997, 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022604783999999999, 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 542 2024-09-14T07:08:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934589+0000", 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1039999999999999e-06 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934594+0000", 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.293e-06 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934612+0000", 2024-09-14T07:08:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8578999999999999e-05 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934626+0000", 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3613e-05 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956738+0000", 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022111927 2024-09-14T07:08:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956951+0000", 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021302100000000001 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957192+0000", 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024124699999999999 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:543 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9201921080000002, 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025291999999999, 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 543 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934646+0000", 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934652+0000", 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000004e-06 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934670+0000", 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.808e-05 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934683+0000", 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3686000000000001e-05 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957298+0000", 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02261471 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957448+0000", 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015033900000000001 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957670+0000", 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022180100000000001 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:544 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.920134483, 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023570109999999998, 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 544 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934704+0000", 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:08:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934726+0000", 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2005999999999999e-05 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934744+0000", 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8297000000000001e-05 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934788+0000", 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3631999999999998e-05 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957813+0000", 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025417999999999 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958049+0000", 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023578 2024-09-14T07:08:48.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958273+0000", 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022392700000000001 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:545 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:08:48.935 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9200213020000003, 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023972594999999999, 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 545 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934817+0000", 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3060000000000001e-06 2024-09-14T07:08:48.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934823+0000", 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-06 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934843+0000", 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9576e-05 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934859+0000", 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.588e-05 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958368+0000", 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023509124999999999 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958487+0000", 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000119123 2024-09-14T07:08:48.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958788+0000", 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030121500000000002 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.940 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:546 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9199205350000002, 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024419039, 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 546 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934918+0000", 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934925+0000", 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9739999999999996e-06 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934946+0000", 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0528999999999999e-05 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934960+0000", 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4343e-05 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958881+0000", 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023921114 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959100+0000", 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021872699999999999 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959336+0000", 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023562600000000001 2024-09-14T07:08:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:547 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9198542270000001, 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024711538000000002, 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 547 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934984+0000", 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.513e-06 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934990+0000", 2024-09-14T07:08:48.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5929999999999996e-06 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935008+0000", 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7802999999999999e-05 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935021+0000", 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3506000000000001e-05 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959426+0000", 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024404250999999998 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959538+0000", 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112534 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959694+0000", 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015633800000000001 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:548 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9197918559999998, 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024957958999999998, 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 548 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935046+0000", 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.147e-06 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935052+0000", 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6130000000000003e-06 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935071+0000", 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8522999999999999e-05 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935084+0000", 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3823e-05 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959754+0000", 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024669673 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959864+0000", 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109757 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.960003+0000", 2024-09-14T07:08:48.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013942300000000001 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:591 3.1f 3:f9b02d91:::rbd_data.1161605482a1.0000000000000004:e [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9188372920000001, 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020703709000000001, 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:48.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 591 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:08:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936001+0000", 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.083e-06 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936006+0000", 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936024+0000", 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7693000000000002e-05 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:48.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936037+0000", 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3303e-05 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956267+0000", 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020229626000000001 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956482+0000", 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000215069 2024-09-14T07:08:48.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956704+0000", 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022182500000000001 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:48.961 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:48.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:48.964 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:48.957+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:49.064 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:49.061+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:49.109 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:49.165 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:49.161+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:49.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:08:49.244 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:49.244 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:49.244 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:49.244 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:49.245 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:49.265 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:49.261+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:49.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:49.366 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:49.361+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:49.454 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:295 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.983283933000003, 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028784614, 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:49.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 295 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:49.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469603+0000", 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469619+0000", 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6163000000000001e-05 2024-09-14T07:08:49.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469646+0000", 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6040000000000001e-05 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469660+0000", 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4843e-05 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488319+0000", 2024-09-14T07:08:49.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018658726 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488567+0000", 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024742500000000001 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.489393+0000", 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082641099999999999 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:49.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493423+0000", 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040299050000000003 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498065+0000", 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046422310000000001 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498254+0000", 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000189278 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498385+0000", 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013043899999999999 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1514 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237089255999997, 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028657183999999999, 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1514 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2300000000000002e-07 2024-09-14T07:08:49.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215804+0000", 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.602e-06 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215811+0000", 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8190000000000003e-06 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215816+0000", 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2229999999999998e-06 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244228+0000", 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028411750999999999 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244350+0000", 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012267199999999999 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244452+0000", 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010169400000000001 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1515 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237061152000003, 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843423999999999, 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1515 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215824+0000", 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215826+0000", 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-06 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215832+0000", 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.947e-06 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215837+0000", 2024-09-14T07:08:49.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6570000000000002e-06 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244479+0000", 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028642223000000001 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244559+0000", 2024-09-14T07:08:49.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9443000000000005e-05 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244666+0000", 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107538 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1517 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10027008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:49.471 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237040620000002, 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029071107999999998, 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1517 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215844+0000", 2024-09-14T07:08:49.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215846+0000", 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.821e-06 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215852+0000", 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215857+0000", 2024-09-14T07:08:49.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6099999999999997e-06 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244719+0000", 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028862291000000002 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244803+0000", 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3523999999999995e-05 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244915+0000", 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111612 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1518 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237019942000003, 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029262857, 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1518 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215865+0000", 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5699999999999998e-07 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215867+0000", 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215878+0000", 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1399e-05 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215883+0000", 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244948+0000", 2024-09-14T07:08:49.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029064834000000001 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245029+0000", 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0693999999999997e-05 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245127+0000", 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8218000000000001e-05 2024-09-14T07:08:49.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1520 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.236992987000001, 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029524788999999999, 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1520 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215892+0000", 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215894+0000", 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6860000000000002e-06 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215902+0000", 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7929999999999998e-06 2024-09-14T07:08:49.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215907+0000", 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245181+0000", 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029273648999999999 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245287+0000", 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010595299999999999 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245416+0000", 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012906300000000001 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1522 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.485 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.236967475999997, 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029733393, 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1522 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:49.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9599999999999999e-07 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215919+0000", 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2749999999999998e-06 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215931+0000", 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215937+0000", 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245451+0000", 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029513475000000001 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245537+0000", 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5975000000000003e-05 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245650+0000", 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011319099999999999 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1523 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13959168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.236934067, 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029960937999999999, 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1523 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.490 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215954+0000", 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3689999999999998e-06 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215961+0000", 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2420000000000003e-06 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215967+0000", 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8329999999999999e-06 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245715+0000", 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029748177000000001 2024-09-14T07:08:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245798+0000", 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3338999999999996e-05 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245911+0000", 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112507 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1524 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.236908671999998, 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030280225000000001, 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1524 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:49.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215976+0000", 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215978+0000", 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0509999999999999e-06 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215985+0000", 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8120000000000002e-06 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215990+0000", 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5729999999999998e-06 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246034+0000", 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030043767999999998 2024-09-14T07:08:49.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246121+0000", 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7093000000000001e-05 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246256+0000", 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013446899999999999 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1525 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.236852098, 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030480007, 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.500 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1525 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216033+0000", 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0800000000000004e-07 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216036+0000", 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8600000000000003e-06 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216045+0000", 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3720000000000005e-06 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216052+0000", 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246311+0000", 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030259496 2024-09-14T07:08:49.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246397+0000", 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5339000000000004e-05 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246512+0000", 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000115393 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1665 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231500279000002, 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028695215, 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1665 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:49.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9699999999999996e-07 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221387+0000", 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221395+0000", 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0399999999999993e-06 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221401+0000", 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.806e-06 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249893+0000", 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028491951000000001 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249971+0000", 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8419999999999998e-05 2024-09-14T07:08:49.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250079+0000", 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010798600000000001 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1667 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231474423999998, 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028871159, 2024-09-14T07:08:49.509 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1667 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221413+0000", 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.396e-06 2024-09-14T07:08:49.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221422+0000", 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2250000000000006e-06 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221428+0000", 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0839999999999997e-06 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250112+0000", 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028684345 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250191+0000", 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8571000000000004e-05 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250281+0000", 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0094999999999997e-05 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1668 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231435005999998, 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02902478, 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1668 2024-09-14T07:08:49.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221450+0000", 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1399999999999997e-07 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221453+0000", 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221462+0000", 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.884e-06 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221468+0000", 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5390000000000004e-06 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250311+0000", 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843160999999999 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250389+0000", 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7550000000000001e-05 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250474+0000", 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4982999999999996e-05 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.518 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1669 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231381712000001, 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029194801999999999, 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1669 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221503+0000", 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221506+0000", 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221516+0000", 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9450000000000005e-06 2024-09-14T07:08:49.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221523+0000", 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250506+0000", 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028983992 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250609+0000", 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010213399999999999 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250697+0000", 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8551e-05 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1670 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231351441999998, 2024-09-14T07:08:49.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029401132999999999, 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1670 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:49.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221536+0000", 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5550000000000001e-06 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221543+0000", 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4939999999999997e-06 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250744+0000", 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029195045999999999 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250836+0000", 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1589999999999996e-05 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250934+0000", 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7826999999999997e-05 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1904 3.b 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9175040~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.163991465000002, 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.101088252, 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1904 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288894+0000", 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.666e-06 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315149+0000", 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026254493 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315177+0000", 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315200+0000", 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2949e-05 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315307+0000", 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107222 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315483+0000", 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017589 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.317437+0000", 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019539890000000002 2024-09-14T07:08:49.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.384167+0000", 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066730085999999994 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389856+0000", 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056887079999999998 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389925+0000", 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8977e-05 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389981+0000", 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6158e-05 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.534 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1907 3.1a e43/21 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head v 43'107, mlcod=42'105)", 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.130706177, 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.068409926999999995, 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8299999999999998e-07 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331614+0000", 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094355310000000005 2024-09-14T07:08:49.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331636+0000", 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1804e-05 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331668+0000", 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2094999999999997e-05 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331759+0000", 2024-09-14T07:08:49.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0901000000000003e-05 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331787+0000", 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8750000000000001e-05 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390536+0000", 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058748732999999997 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390588+0000", 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1829999999999997e-05 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1911 3.8 e43/25 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head v 43'64, mlcod=42'61)", 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.103484948000002, 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.060945845999999998, 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:49.539 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349401+0000", 2024-09-14T07:08:49.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.846e-06 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354404+0000", 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005003146 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354427+0000", 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2617999999999999e-05 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354440+0000", 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3559e-05 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354525+0000", 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4986000000000003e-05 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354551+0000", 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5276999999999999e-05 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410278+0000", 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055727499999999999 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410345+0000", 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6914000000000003e-05 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:08:49.543 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.773099096999999, 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:49.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:08:49.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:08:49.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:08:49.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:08:49.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.549 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.549 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:1096 3.b 3:d2fcf993:::rbd_data.1161605482a1.0000000000000009:e [sparse-read 4259840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:49.549 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.998065+0000", 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.454819337, 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030098985000000002, 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:49.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:49.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1096 2024-09-14T07:08:49.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:49.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:49.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998065+0000", 2024-09-14T07:08:49.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998065+0000", 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:49.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998066+0000", 2024-09-14T07:08:49.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:08:49.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:49.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998081+0000", 2024-09-14T07:08:49.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5048999999999999e-05 2024-09-14T07:08:49.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:49.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998096+0000", 2024-09-14T07:08:49.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5255e-05 2024-09-14T07:08:49.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:49.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998109+0000", 2024-09-14T07:08:49.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2788e-05 2024-09-14T07:08:49.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:44.009825+0000", 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011716379000000001 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:44.010043+0000", 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021744399999999999 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:49.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:44.028164+0000", 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018121035000000001 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:49.557 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:49.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:49.559 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:49.553+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:49.659 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:49.657+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:49.724 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:49.724 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:49.724 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:49.724 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:49.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:49.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:49.760 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:49.757+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:49.862 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:49.857+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:49.878 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:49.895 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:49.895 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:49.903 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:49.903 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:49.913 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:49.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:49.935 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:49.935 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:49.956 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:49.956 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:49.962 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:49.957+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:49.977 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:49.978 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:50.035 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:50.035 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:50.050 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:50.095 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:50.093+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:50.163 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:50.161+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:50.182 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:50.183 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:50.192 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:08:50.196 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:50.220 INFO:teuthology.orchestra.run.smithi145.stdout:1796 punch from 0x4faa00 to 0x503200, (0x8800 bytes) 2024-09-14T07:08:50.223 INFO:teuthology.orchestra.run.smithi145.stdout:1797 read 0x46d600 thru 0x46fbe7 (0x25e8 bytes) 2024-09-14T07:08:50.252 INFO:teuthology.orchestra.run.smithi145.stdout:1798 trunc from 0x1f6da00 to 0xd2c6e00 2024-09-14T07:08:50.265 INFO:teuthology.orchestra.run.smithi145.stdout:1799 write 0x99c400 thru 0x99f459 (0x305a bytes) 2024-09-14T07:08:50.268 INFO:teuthology.orchestra.run.smithi145.stdout:1801 read 0xc941a00 thru 0xc94a431 (0x8a32 bytes) 2024-09-14T07:08:50.283 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:50.281+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:50.303 INFO:teuthology.orchestra.run.smithi145.stdout:1802 write 0xed92200 thru 0xed93eb4 (0x1cb5 bytes) 2024-09-14T07:08:50.304 INFO:teuthology.orchestra.run.smithi145.stdout:1804 write 0x87c400 thru 0x889d0b (0xd90c bytes) 2024-09-14T07:08:50.306 INFO:teuthology.orchestra.run.smithi145.stdout:1805 punch from 0x8bd4a00 to 0x8bd8600, (0x3c00 bytes) 2024-09-14T07:08:50.306 INFO:teuthology.orchestra.run.smithi145.stdout:1809 punch from 0x4748200 to 0x4751800, (0x9600 bytes) 2024-09-14T07:08:50.307 INFO:teuthology.orchestra.run.smithi145.stdout:1811 write 0x1e2ae00 thru 0x1e30e11 (0x6012 bytes) 2024-09-14T07:08:50.326 INFO:teuthology.orchestra.run.smithi145.stdout:1812 trunc from 0xed94000 to 0x58ec600 2024-09-14T07:08:50.364 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:50.361+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:50.376 INFO:teuthology.orchestra.run.smithi145.stdout:1813 punch from 0xe24800 to 0xe32400, (0xdc00 bytes) 2024-09-14T07:08:50.377 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:50.377 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:50.378 INFO:teuthology.orchestra.run.smithi145.stdout:1816 punch from 0x25e3800 to 0x25eb400, (0x7c00 bytes) 2024-09-14T07:08:50.395 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:50.397 INFO:teuthology.orchestra.run.smithi145.stdout:1817 write 0x771d400 thru 0x771dfe4 (0xbe5 bytes) 2024-09-14T07:08:50.410 INFO:teuthology.orchestra.run.smithi145.stdout:1818 trunc from 0x771e000 to 0x308a800 2024-09-14T07:08:50.464 INFO:teuthology.orchestra.run.smithi145.stdout:1822 write 0x944b000 thru 0x944bb41 (0xb42 bytes) 2024-09-14T07:08:50.465 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:50.461+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:50.466 INFO:teuthology.orchestra.run.smithi145.stdout:1828 read 0x4f72600 thru 0x4f77436 (0x4e37 bytes) 2024-09-14T07:08:50.467 INFO:teuthology.orchestra.run.smithi145.stdout:1829 read 0x102c200 thru 0x1039600 (0xd401 bytes) 2024-09-14T07:08:50.469 INFO:teuthology.orchestra.run.smithi145.stdout:1831 punch from 0xcc0600 to 0xccf600, (0xf000 bytes) 2024-09-14T07:08:50.470 INFO:teuthology.orchestra.run.smithi145.stdout:1833 punch from 0x81bda00 to 0x81c5400, (0x7a00 bytes) 2024-09-14T07:08:50.470 INFO:teuthology.orchestra.run.smithi145.stdout:1834 write 0x81da600 thru 0x81e7c69 (0xd66a bytes) 2024-09-14T07:08:50.478 INFO:teuthology.orchestra.run.smithi145.stdout:1838 trunc from 0x944bc00 to 0xae41200 2024-09-14T07:08:50.526 INFO:teuthology.orchestra.run.smithi145.stdout:1839 trunc from 0xae41200 to 0x1764600 2024-09-14T07:08:50.565 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:50.561+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:50.576 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:50.576 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:50.589 INFO:teuthology.orchestra.run.smithi145.stdout:1845 write 0x2387200 thru 0x2394ed3 (0xdcd4 bytes) 2024-09-14T07:08:50.592 INFO:teuthology.orchestra.run.smithi145.stdout:1849 read 0x21f2800 thru 0x21fe390 (0xbb91 bytes) 2024-09-14T07:08:50.593 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:50.602 INFO:teuthology.orchestra.run.smithi145.stdout:1852 trunc from 0x2395000 to 0x57e2a00 2024-09-14T07:08:50.621 INFO:teuthology.orchestra.run.smithi145.stdout:1853 punch from 0xd7de00 to 0xd82400, (0x4600 bytes) 2024-09-14T07:08:50.622 INFO:teuthology.orchestra.run.smithi145.stdout:1854 read 0x2a8b800 thru 0x2a8eaec (0x32ed bytes) 2024-09-14T07:08:50.644 INFO:teuthology.orchestra.run.smithi145.stdout:1855 write 0x6fb3000 thru 0x6fb71d6 (0x41d7 bytes) 2024-09-14T07:08:50.652 INFO:teuthology.orchestra.run.smithi145.stdout:1856 trunc from 0x6fb7200 to 0x4c5e400 2024-09-14T07:08:50.677 INFO:teuthology.orchestra.run.smithi145.stdout:1859 punch from 0x1168600 to 0x1169200, (0xc00 bytes) 2024-09-14T07:08:50.678 INFO:teuthology.orchestra.run.smithi145.stdout:1860 read 0x2bb5800 thru 0x2bbbb12 (0x6313 bytes) 2024-09-14T07:08:50.683 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:50.681+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:50.712 INFO:teuthology.orchestra.run.smithi145.stdout:1861 write 0xd66c000 thru 0xd672194 (0x6195 bytes) 2024-09-14T07:08:50.713 INFO:teuthology.orchestra.run.smithi145.stdout:1862 write 0x3ff600 thru 0x40c553 (0xcf54 bytes) 2024-09-14T07:08:50.715 INFO:teuthology.orchestra.run.smithi145.stdout:1863 read 0x178000 thru 0x17d8df (0x58e0 bytes) 2024-09-14T07:08:50.716 INFO:teuthology.orchestra.run.smithi145.stdout:1866 write 0xbbf3400 thru 0xbc0334b (0xff4c bytes) 2024-09-14T07:08:50.717 INFO:teuthology.orchestra.run.smithi145.stdout:1867 read 0x310f000 thru 0x3110bed (0x1bee bytes) 2024-09-14T07:08:50.736 INFO:teuthology.orchestra.run.smithi145.stdout:1868 trunc from 0xd672200 to 0x5701600 2024-09-14T07:08:50.767 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:50.761+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:50.774 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:50.775 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:50.791 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:50.791 INFO:teuthology.orchestra.run.smithi145.stdout:1870 read 0x3692400 thru 0x36a2210 (0xfe11 bytes) 2024-09-14T07:08:50.793 INFO:teuthology.orchestra.run.smithi145.stdout:1872 write 0x3836a00 thru 0x38396d0 (0x2cd1 bytes) 2024-09-14T07:08:50.795 INFO:teuthology.orchestra.run.smithi145.stdout:1875 write 0x4feb200 thru 0x4ff3ee9 (0x8cea bytes) 2024-09-14T07:08:50.796 INFO:teuthology.orchestra.run.smithi145.stdout:1880 punch from 0x1d5d200 to 0x1d5f400, (0x2200 bytes) 2024-09-14T07:08:50.819 INFO:teuthology.orchestra.run.smithi145.stdout:1882 write 0x7599400 thru 0x759c033 (0x2c34 bytes) 2024-09-14T07:08:50.822 INFO:teuthology.orchestra.run.smithi145.stdout:1886 read 0x6631e00 thru 0x66388c0 (0x6ac1 bytes) 2024-09-14T07:08:50.838 INFO:teuthology.orchestra.run.smithi145.stdout:1888 trunc from 0x759c200 to 0x21cd400 2024-09-14T07:08:50.875 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:50.873+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:50.886 INFO:teuthology.orchestra.run.smithi145.stdout:1889 write 0x320d000 thru 0x32164e9 (0x94ea bytes) 2024-09-14T07:08:50.888 INFO:teuthology.orchestra.run.smithi145.stdout:1890 punch from 0x3060c00 to 0x3068a00, (0x7e00 bytes) 2024-09-14T07:08:50.902 INFO:teuthology.orchestra.run.smithi145.stdout:1893 trunc from 0x3216600 to 0xd5d400 2024-09-14T07:08:50.969 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:50.969 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:50.970 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:50.965+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:50.974 INFO:teuthology.orchestra.run.smithi145.stdout:1895 write 0xc80f000 thru 0xc81b999 (0xc99a bytes) 2024-09-14T07:08:50.975 INFO:teuthology.orchestra.run.smithi145.stdout:1898 read 0x5984400 thru 0x598efe2 (0xabe3 bytes) 2024-09-14T07:08:50.976 INFO:teuthology.orchestra.run.smithi145.stdout:1899 write 0x25e1400 thru 0x25e61ad (0x4dae bytes) 2024-09-14T07:08:50.978 INFO:teuthology.orchestra.run.smithi145.stdout:1901 punch from 0xbb6c00 to 0xbb9e00, (0x3200 bytes) 2024-09-14T07:08:50.979 INFO:teuthology.orchestra.run.smithi145.stdout:1903 punch from 0x3c03800 to 0x3c07200, (0x3a00 bytes) 2024-09-14T07:08:50.979 INFO:teuthology.orchestra.run.smithi145.stdout:1904 read 0x5285c00 thru 0x5292355 (0xc756 bytes) 2024-09-14T07:08:50.984 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:50.985 INFO:teuthology.orchestra.run.smithi145.stdout:1905 read 0x8629000 thru 0x862b821 (0x2822 bytes) 2024-09-14T07:08:50.985 INFO:teuthology.orchestra.run.smithi145.stdout:1907 punch from 0x5d9b000 to 0x5d9f600, (0x4600 bytes) 2024-09-14T07:08:51.003 INFO:teuthology.orchestra.run.smithi145.stdout:1909 write 0xdfdca00 thru 0xdfe55c8 (0x8bc9 bytes) 2024-09-14T07:08:51.004 INFO:teuthology.orchestra.run.smithi145.stdout:1911 read 0xcfea000 thru 0xcfeccea (0x2ceb bytes) 2024-09-14T07:08:51.004 INFO:teuthology.orchestra.run.smithi145.stdout:1912 read 0x65fe800 thru 0x6607eaf (0x96b0 bytes) 2024-09-14T07:08:51.006 INFO:teuthology.orchestra.run.smithi145.stdout:1913 read 0xb9b9c00 thru 0xb9c68e8 (0xcce9 bytes) 2024-09-14T07:08:51.007 INFO:teuthology.orchestra.run.smithi145.stdout:1914 read 0x4406600 thru 0x440b80f (0x5210 bytes) 2024-09-14T07:08:51.008 INFO:teuthology.orchestra.run.smithi145.stdout:1916 write 0x4a76c00 thru 0x4a79a50 (0x2e51 bytes) 2024-09-14T07:08:51.033 INFO:teuthology.orchestra.run.smithi145.stdout:1918 trunc from 0xdfe5600 to 0x138a200 2024-09-14T07:08:51.070 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:51.065+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:51.079 INFO:teuthology.orchestra.run.smithi145.stdout:1919 read 0x10f4c00 thru 0x10f813a (0x353b bytes) 2024-09-14T07:08:51.082 INFO:teuthology.orchestra.run.smithi145.stdout:1923 read 0x1174600 thru 0x117bdb8 (0x77b9 bytes) 2024-09-14T07:08:51.083 INFO:teuthology.orchestra.run.smithi145.stdout:1925 read 0x46b600 thru 0x46b912 (0x313 bytes) 2024-09-14T07:08:51.084 INFO:teuthology.orchestra.run.smithi145.stdout:1927 punch from 0x12b2000 to 0x12b4a00, (0x2a00 bytes) 2024-09-14T07:08:51.084 INFO:teuthology.orchestra.run.smithi145.stdout:skipping zero length punch hole 2024-09-14T07:08:51.084 INFO:teuthology.orchestra.run.smithi145.stdout:1933 read 0x6a6e00 thru 0x6b1df2 (0xaff3 bytes) 2024-09-14T07:08:51.088 INFO:teuthology.orchestra.run.smithi145.stdout:1934 read 0x480400 thru 0x481a14 (0x1615 bytes) 2024-09-14T07:08:51.089 INFO:teuthology.orchestra.run.smithi145.stdout:1935 punch from 0x8c9c00 to 0x8d0800, (0x6c00 bytes) 2024-09-14T07:08:51.089 INFO:teuthology.orchestra.run.smithi145.stdout:1938 read 0x109c600 thru 0x10a1f8d (0x598e bytes) 2024-09-14T07:08:51.090 INFO:teuthology.orchestra.run.smithi145.stdout:1941 read 0x416200 thru 0x417351 (0x1152 bytes) 2024-09-14T07:08:51.120 INFO:teuthology.orchestra.run.smithi145.stdout:1942 write 0x66c0e00 thru 0x66d08a1 (0xfaa2 bytes) 2024-09-14T07:08:51.122 INFO:teuthology.orchestra.run.smithi145.stdout:1944 read 0x3493800 thru 0x349de97 (0xa698 bytes) 2024-09-14T07:08:51.123 INFO:teuthology.orchestra.run.smithi145.stdout:1951 read 0x4d72200 thru 0x4d7fb57 (0xd958 bytes) 2024-09-14T07:08:51.124 INFO:teuthology.orchestra.run.smithi145.stdout:1953 read 0x3e0a200 thru 0x3e0e0d7 (0x3ed8 bytes) 2024-09-14T07:08:51.124 INFO:teuthology.orchestra.run.smithi145.stdout:1954 read 0x5d09000 thru 0x5d117bc (0x87bd bytes) 2024-09-14T07:08:51.150 INFO:teuthology.orchestra.run.smithi145.stdout:1955 write 0x7ec1e00 thru 0x7ec4601 (0x2802 bytes) 2024-09-14T07:08:51.153 INFO:teuthology.orchestra.run.smithi145.stdout:1957 read 0x61b0200 thru 0x61bf14b (0xef4c bytes) 2024-09-14T07:08:51.156 INFO:teuthology.orchestra.run.smithi145.stdout:1960 punch from 0xfb6600 to 0xfc4800, (0xe200 bytes) 2024-09-14T07:08:51.156 INFO:teuthology.orchestra.run.smithi145.stdout:1961 read 0x2afce00 thru 0x2b006d8 (0x38d9 bytes) 2024-09-14T07:08:51.158 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:51.158 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:51.159 INFO:teuthology.orchestra.run.smithi145.stdout:1963 read 0x681ca00 thru 0x68208e5 (0x3ee6 bytes) 2024-09-14T07:08:51.161 INFO:teuthology.orchestra.run.smithi145.stdout:1964 write 0x2cba400 thru 0x2cc1499 (0x709a bytes) 2024-09-14T07:08:51.163 INFO:teuthology.orchestra.run.smithi145.stdout:1965 punch from 0x28aee00 to 0x28b8a00, (0x9c00 bytes) 2024-09-14T07:08:51.171 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:51.165+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:51.174 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:51.188 INFO:teuthology.orchestra.run.smithi145.stdout:1966 write 0xce05e00 thru 0xce1141c (0xb61d bytes) 2024-09-14T07:08:51.190 INFO:teuthology.orchestra.run.smithi145.stdout:1968 write 0x1a17c00 thru 0x1a2225b (0xa65c bytes) 2024-09-14T07:08:51.191 INFO:teuthology.orchestra.run.smithi145.stdout:1970 read 0x808fa00 thru 0x809eb0c (0xf10d bytes) 2024-09-14T07:08:51.192 INFO:teuthology.orchestra.run.smithi145.stdout:1972 write 0xaa60000 thru 0xaa645d3 (0x45d4 bytes) 2024-09-14T07:08:51.193 INFO:teuthology.orchestra.run.smithi145.stdout:1973 read 0x7373c00 thru 0x737f715 (0xbb16 bytes) 2024-09-14T07:08:51.194 INFO:teuthology.orchestra.run.smithi145.stdout:1974 punch from 0x38ac800 to 0x38b8c00, (0xc400 bytes) 2024-09-14T07:08:51.195 INFO:teuthology.orchestra.run.smithi145.stdout:1975 write 0x64c1c00 thru 0x64c3ac1 (0x1ec2 bytes) 2024-09-14T07:08:51.196 INFO:teuthology.orchestra.run.smithi145.stdout:1977 write 0x917ee00 thru 0x9183d90 (0x4f91 bytes) 2024-09-14T07:08:51.198 INFO:teuthology.orchestra.run.smithi145.stdout:1978 read 0xa8c7200 thru 0xa8cf742 (0x8543 bytes) 2024-09-14T07:08:51.199 INFO:teuthology.orchestra.run.smithi145.stdout:1979 write 0x8bbf000 thru 0x8bc8f80 (0x9f81 bytes) 2024-09-14T07:08:51.199 INFO:teuthology.orchestra.run.smithi145.stdout:1982 clone 14 order 21 su 16384 sc 14 2024-09-14T07:08:51.271 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:51.265+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:51.338 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:51.338 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:51.351 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:51.372 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:51.369+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:51.473 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:51.469+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:51.519 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:51.519 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:51.533 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:51.576 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:51.573+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:51.675 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:51.669+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:51.701 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:51.701 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:51.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:51.776 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:51.773+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:51.850 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:51.850 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:51.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:51.907 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:51.905+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:51.978 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:51.973+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:51.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:52.028 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:52.028 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:52.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:52.087 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:52.085+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:52.142 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:52.142 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:52.159 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:52.159 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:52.176 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:52.176 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:52.179 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:52.177+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:52.197 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:52.197 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 34.923585181999997, 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:08:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:08:52.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:08:52.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 34.894077551000002, 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:52.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 34.885430192000001, 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.223 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:08:52.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:08:52.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.219337851999999, 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:52.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 30.052608626000001, 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:08:52.236 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.870677111999999, 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:08:52.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:08:52.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.119607618, 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:08:52.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:08:52.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4732.0:183 3.1b 3:d96d4dd6:::rbd_data.1161bf39a192.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 6709248~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.269737578000001, 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030801473999999999, 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4732", 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/303097868", 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 183 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:08:52.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942590+0000", 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944540+0000", 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001949824 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944564+0000", 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4199e-05 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944574+0000", 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944644+0000", 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0240999999999994e-05 2024-09-14T07:08:52.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944798+0000", 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153432 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.945082+0000", 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028403899999999998 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.959401+0000", 2024-09-14T07:08:52.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014319159999999999 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973083+0000", 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013682016 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973261+0000", 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017752999999999999 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973390+0000", 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012949500000000001 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:538 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.278051949, 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020569724000000001, 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.252 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 538 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934275+0000", 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934281+0000", 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934299+0000", 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8470000000000001e-05 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934313+0000", 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3196e-05 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954450+0000", 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020137616000000001 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954629+0000", 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017857499999999999 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954844+0000", 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021515099999999999 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:539 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:08:52.256 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.277930199, 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02104733, 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 539 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:08:52.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934398+0000", 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.813e-06 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934404+0000", 2024-09-14T07:08:52.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7460000000000003e-06 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934425+0000", 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.033e-05 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934440+0000", 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5019e-05 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954916+0000", 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020476173 2024-09-14T07:08:52.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955086+0000", 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169655 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955443+0000", 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035759399999999998 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:540 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2778626039999992, 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021617992999999999, 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 540 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934465+0000", 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4130000000000001e-06 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934471+0000", 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7259999999999999e-06 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934489+0000", 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8263000000000001e-05 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934506+0000", 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955539+0000", 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021033046999999999 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955813+0000", 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00027412200000000002 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956082+0000", 2024-09-14T07:08:52.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000268829 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:541 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2777976659999997, 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022127142999999998, 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 541 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934530+0000", 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.037e-06 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934535+0000", 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8259999999999998e-06 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934554+0000", 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934567+0000", 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3573e-05 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956183+0000", 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021615059999999998 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956395+0000", 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021209399999999999 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956656+0000", 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026098000000000001 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:542 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2777386750000002, 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022604783999999999, 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 542 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934589+0000", 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1039999999999999e-06 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934594+0000", 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.293e-06 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934612+0000", 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8578999999999999e-05 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934626+0000", 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3613e-05 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956738+0000", 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022111927 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956951+0000", 2024-09-14T07:08:52.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021302100000000001 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957192+0000", 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024124699999999999 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:543 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2776811329999997, 2024-09-14T07:08:52.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025291999999999, 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 543 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934646+0000", 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934652+0000", 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000004e-06 2024-09-14T07:08:52.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934670+0000", 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.808e-05 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934683+0000", 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3686000000000001e-05 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957298+0000", 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02261471 2024-09-14T07:08:52.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957448+0000", 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015033900000000001 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957670+0000", 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022180100000000001 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.277 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:544 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2776235079999996, 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023570109999999998, 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 544 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934704+0000", 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934726+0000", 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2005999999999999e-05 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934744+0000", 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8297000000000001e-05 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934788+0000", 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3631999999999998e-05 2024-09-14T07:08:52.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957813+0000", 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025417999999999 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958049+0000", 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023578 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958273+0000", 2024-09-14T07:08:52.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022392700000000001 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:545 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2775103269999999, 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023972594999999999, 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 545 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934817+0000", 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3060000000000001e-06 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934823+0000", 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-06 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934843+0000", 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9576e-05 2024-09-14T07:08:52.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934859+0000", 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.588e-05 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958368+0000", 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023509124999999999 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958487+0000", 2024-09-14T07:08:52.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000119123 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958788+0000", 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030121500000000002 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:546 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2774095600000006, 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024419039, 2024-09-14T07:08:52.286 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 546 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934918+0000", 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934925+0000", 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9739999999999996e-06 2024-09-14T07:08:52.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934946+0000", 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0528999999999999e-05 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934960+0000", 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4343e-05 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958881+0000", 2024-09-14T07:08:52.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023921114 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959100+0000", 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021872699999999999 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959336+0000", 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023562600000000001 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:547 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2773432519999997, 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024711538000000002, 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 547 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934984+0000", 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.513e-06 2024-09-14T07:08:52.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934990+0000", 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5929999999999996e-06 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935008+0000", 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7802999999999999e-05 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935021+0000", 2024-09-14T07:08:52.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3506000000000001e-05 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959426+0000", 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024404250999999998 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959538+0000", 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112534 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959694+0000", 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015633800000000001 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:548 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2772808809999994, 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024957958999999998, 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.295 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 548 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935046+0000", 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.147e-06 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935052+0000", 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6130000000000003e-06 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935071+0000", 2024-09-14T07:08:52.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8522999999999999e-05 2024-09-14T07:08:52.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935084+0000", 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3823e-05 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959754+0000", 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024669673 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959864+0000", 2024-09-14T07:08:52.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109757 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.960003+0000", 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013942300000000001 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:591 3.1f 3:f9b02d91:::rbd_data.1161605482a1.0000000000000004:e [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:08:52.302 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2763263170000005, 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020703709000000001, 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 591 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936001+0000", 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.083e-06 2024-09-14T07:08:52.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936006+0000", 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936024+0000", 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7693000000000002e-05 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936037+0000", 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3303e-05 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956267+0000", 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020229626000000001 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956482+0000", 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000215069 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956704+0000", 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022182500000000001 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.307 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:52.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:08:52.309 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:52.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:52.311 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:52.312 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:52.305+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:52.383 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone13 from 0xce11600 (overlap 0xd5d400) to 0x0 2024-09-14T07:08:52.410 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:52.405+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:52.489 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:52.489 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:52.489 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:52.489 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:52.505 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:52.511 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:52.509+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:52.612 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:52.609+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:52.623 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:52.623 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:52.623 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:52.623 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:52.623 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:52.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:52.714 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:52.709+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:52.805 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:52.805 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4558.0:295 3.b 3:d1d1c008:::rbd_object_map.1161dbabc1ae:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 44.335505406999999, 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028784614, 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4558", 2024-09-14T07:08:52.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/970740106", 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 295 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469600+0000", 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469603+0000", 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469619+0000", 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6163000000000001e-05 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469646+0000", 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6040000000000001e-05 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.469660+0000", 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4843e-05 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488319+0000", 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018658726 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.488567+0000", 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024742500000000001 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.489393+0000", 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082641099999999999 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.493423+0000", 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040299050000000003 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498065+0000", 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046422310000000001 2024-09-14T07:08:52.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498254+0000", 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000189278 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:08.498385+0000", 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013043899999999999 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.811 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1514 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 7274496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589310730000001, 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028657183999999999, 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1514 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215795+0000", 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2300000000000002e-07 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215804+0000", 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.602e-06 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215811+0000", 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8190000000000003e-06 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215816+0000", 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2229999999999998e-06 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244228+0000", 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028411750999999999 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244350+0000", 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012267199999999999 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244452+0000", 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010169400000000001 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1515 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 9437184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589282625999999, 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843423999999999, 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1515 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215823+0000", 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215824+0000", 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215826+0000", 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9939999999999999e-06 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215832+0000", 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.947e-06 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215837+0000", 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6570000000000002e-06 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244479+0000", 2024-09-14T07:08:52.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028642223000000001 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244559+0000", 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9443000000000005e-05 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244666+0000", 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107538 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.819 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1517 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10027008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589262093999999, 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029071107999999998, 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1517 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215843+0000", 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215844+0000", 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215846+0000", 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.821e-06 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215852+0000", 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215857+0000", 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6099999999999997e-06 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244719+0000", 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028862291000000002 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244803+0000", 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3523999999999995e-05 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244915+0000", 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111612 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1518 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10616832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589241416, 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029262857, 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1518 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215864+0000", 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215865+0000", 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5699999999999998e-07 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215867+0000", 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215878+0000", 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1399e-05 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215883+0000", 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.244948+0000", 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029064834000000001 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245029+0000", 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0693999999999997e-05 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245127+0000", 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8218000000000001e-05 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1520 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 11206656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589214460999997, 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029524788999999999, 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1520 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.828 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215891+0000", 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215892+0000", 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215894+0000", 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6860000000000002e-06 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215902+0000", 2024-09-14T07:08:52.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7929999999999998e-06 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215907+0000", 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245181+0000", 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029273648999999999 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245287+0000", 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010595299999999999 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245416+0000", 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012906300000000001 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1522 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13369344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:52.832 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.58918895, 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029733393, 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1522 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:52.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215917+0000", 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9599999999999999e-07 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215919+0000", 2024-09-14T07:08:52.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2749999999999998e-06 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215931+0000", 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215937+0000", 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245451+0000", 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029513475000000001 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245537+0000", 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5975000000000003e-05 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245650+0000", 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011319099999999999 2024-09-14T07:08:52.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1523 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13959168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589155540999997, 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029960937999999999, 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1523 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215950+0000", 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215954+0000", 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3689999999999998e-06 2024-09-14T07:08:52.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215961+0000", 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2420000000000003e-06 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215967+0000", 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8329999999999999e-06 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245715+0000", 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029748177000000001 2024-09-14T07:08:52.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245798+0000", 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3338999999999996e-05 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.245911+0000", 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112507 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1524 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14548992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589130146000002, 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030280225000000001, 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1524 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215975+0000", 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215976+0000", 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215978+0000", 2024-09-14T07:08:52.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0509999999999999e-06 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215985+0000", 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8120000000000002e-06 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.215990+0000", 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5729999999999998e-06 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246034+0000", 2024-09-14T07:08:52.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030043767999999998 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246121+0000", 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7093000000000001e-05 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246256+0000", 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013446899999999999 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1525 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 15138816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.589073571999997, 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030480007, 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1525 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.846 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216032+0000", 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216033+0000", 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0800000000000004e-07 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216036+0000", 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8600000000000003e-06 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216045+0000", 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3720000000000005e-06 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.216052+0000", 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246311+0000", 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030259496 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246397+0000", 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5339000000000004e-05 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.246512+0000", 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000115393 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1665 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 10682368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.583721752999999, 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028695215, 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1665 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221384+0000", 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9699999999999996e-07 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221387+0000", 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221395+0000", 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0399999999999993e-06 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221401+0000", 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.806e-06 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249893+0000", 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028491951000000001 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.249971+0000", 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8419999999999998e-05 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250079+0000", 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010798600000000001 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1667 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:52.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.583695898000002, 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028871159, 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1667 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221410+0000", 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221413+0000", 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.396e-06 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221422+0000", 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2250000000000006e-06 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221428+0000", 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0839999999999997e-06 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250112+0000", 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028684345 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250191+0000", 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8571000000000004e-05 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250281+0000", 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0094999999999997e-05 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1668 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 13434880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.583656480000002, 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02902478, 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1668 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.859 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221449+0000", 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221450+0000", 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1399999999999997e-07 2024-09-14T07:08:52.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221453+0000", 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221462+0000", 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.884e-06 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221468+0000", 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5390000000000004e-06 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250311+0000", 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028843160999999999 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250389+0000", 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7550000000000001e-05 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250474+0000", 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4982999999999996e-05 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1669 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.583603185999998, 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029194801999999999, 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.863 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1669 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221502+0000", 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221503+0000", 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221506+0000", 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221516+0000", 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9450000000000005e-06 2024-09-14T07:08:52.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221523+0000", 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250506+0000", 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028983992 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250609+0000", 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010213399999999999 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250697+0000", 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8551e-05 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1670 3.19 3:9e872ec0:::rbd_data.116167e24a2d.0000000000000005:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.583572916000001, 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029401132999999999, 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.867 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1670 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221533+0000", 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221536+0000", 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5550000000000001e-06 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221543+0000", 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4939999999999997e-06 2024-09-14T07:08:52.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250744+0000", 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029195045999999999 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250836+0000", 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1589999999999996e-05 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.250934+0000", 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7826999999999997e-05 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1904 3.b 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9175040~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:52.871 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.516212938999999, 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.101088252, 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1904 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288894+0000", 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.666e-06 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315149+0000", 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026254493 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315177+0000", 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315200+0000", 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2949e-05 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315307+0000", 2024-09-14T07:08:52.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107222 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315483+0000", 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017589 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.317437+0000", 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019539890000000002 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.384167+0000", 2024-09-14T07:08:52.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066730085999999994 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389856+0000", 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056887079999999998 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389925+0000", 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8977e-05 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389981+0000", 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6158e-05 2024-09-14T07:08:52.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1907 3.1a e43/21 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head v 43'107, mlcod=42'105)", 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.482927650999997, 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.068409926999999995, 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.322178+0000", 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8299999999999998e-07 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331614+0000", 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094355310000000005 2024-09-14T07:08:52.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331636+0000", 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1804e-05 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331668+0000", 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2094999999999997e-05 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331759+0000", 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0901000000000003e-05 2024-09-14T07:08:52.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.331787+0000", 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8750000000000001e-05 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390536+0000", 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058748732999999997 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.390588+0000", 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1829999999999997e-05 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1911 3.8 e43/25 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head v 43'64, mlcod=42'61)", 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.455706421999999, 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.060945845999999998, 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349399+0000", 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.349401+0000", 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.846e-06 2024-09-14T07:08:52.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354404+0000", 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005003146 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354427+0000", 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2617999999999999e-05 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354440+0000", 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3559e-05 2024-09-14T07:08:52.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354525+0000", 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4986000000000003e-05 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.354551+0000", 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5276999999999999e-05 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410278+0000", 2024-09-14T07:08:52.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055727499999999999 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.410345+0000", 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6914000000000003e-05 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:52.885 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.125320571, 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:08:52.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.889 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:1096 3.b 3:d2fcf993:::rbd_data.1161605482a1.0000000000000009:e [sparse-read 4259840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.998065+0000", 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8070408110000002, 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030098985000000002, 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1096 2024-09-14T07:08:52.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998065+0000", 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998065+0000", 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:52.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998066+0000", 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998081+0000", 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5048999999999999e-05 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998096+0000", 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5255e-05 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.998109+0000", 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2788e-05 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:44.009825+0000", 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011716379000000001 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:44.010043+0000", 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021744399999999999 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:52.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:44.028164+0000", 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018121035000000001 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:52.895 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T07:08:52.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:52.896 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:52.893+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:52.996 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:52.993+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:53.082 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:53.082 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:53.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.082 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4954.0:803 3.18 3.316319f8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-14T07:08:53.082 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.082554+0000", 2024-09-14T07:08:53.082 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00046654900000000002, 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000484372, 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4954", 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1000935916", 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 803 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.082554+0000", 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.082554+0000", 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.082556+0000", 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.46e-06 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.082563+0000", 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1099999999999997e-06 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.082606+0000", 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2828999999999999e-05 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.082624+0000", 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8598999999999999e-05 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.085 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:08:53.086 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:08:53.086 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:53.097 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:53.093+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:53.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:53.198 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:53.193+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:53.225 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:53.225 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:08:53.225 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-14T07:08:53.226 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:08:53.226 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:08:53.226 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-14T07:08:53.237 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:08:53.272 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:53.273 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:53.273 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:08:53.273 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:08:53.273 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:53.288 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:53.299 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:53.293+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:53.361 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T07:08:53.361 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:53.388 INFO:teuthology.orchestra.run.smithi145.stdout:1983 read 0x570c00 thru 0x57c207 (0xb608 bytes) 2024-09-14T07:08:53.391 INFO:teuthology.orchestra.run.smithi145.stdout:1984 punch from 0x82ebe00 to 0x82eda00, (0x1c00 bytes) 2024-09-14T07:08:53.419 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:53.417+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:53.432 INFO:teuthology.orchestra.run.smithi145.stdout:1985 read 0x57a6800 thru 0x57abb9b (0x539c bytes) 2024-09-14T07:08:53.433 INFO:teuthology.orchestra.run.smithi145.stdout:1986 write 0xc5a1a00 thru 0xc5a787e (0x5e7f bytes) 2024-09-14T07:08:53.465 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1538 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 11534336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242234699000001, 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054129517000000002, 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1538 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220946+0000", 2024-09-14T07:08:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220949+0000", 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-06 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220959+0000", 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0932e-05 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220965+0000", 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.49e-06 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.274976+0000", 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054011221999999998 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275019+0000", 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2889000000000002e-05 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275076+0000", 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6455000000000001e-05 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1539 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242207983, 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054229843, 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1539 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220973+0000", 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220976+0000", 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-06 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1069999999999999e-06 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220988+0000", 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0119999999999996e-06 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275106+0000", 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054118053999999999 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275149+0000", 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2830000000000002e-05 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275203+0000", 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4044e-05 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1540 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242185667000001, 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05433267, 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1540 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220995+0000", 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220996+0000", 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.220998+0000", 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9759999999999998e-06 2024-09-14T07:08:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221009+0000", 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.242e-06 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275233+0000", 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054223461000000001 2024-09-14T07:08:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275275+0000", 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2726999999999999e-05 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275328+0000", 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2332e-05 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1541 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242163470999998, 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054437230000000003, 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1541 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221017+0000", 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221018+0000", 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2e-07 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221020+0000", 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3530000000000002e-06 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221026+0000", 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3110000000000002e-06 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221031+0000", 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275358+0000", 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054326722000000001 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275401+0000", 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2849000000000003e-05 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275454+0000", 2024-09-14T07:08:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3844000000000002e-05 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1542 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 15466496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242143057, 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054541385999999997, 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1542 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221038+0000", 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3700000000000001e-07 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221040+0000", 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8059999999999999e-06 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221046+0000", 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1009999999999997e-06 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221051+0000", 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6940000000000003e-06 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275484+0000", 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054433003000000001 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275527+0000", 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3031999999999997e-05 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275579+0000", 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2413000000000001e-05 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1543 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16056320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242123300999999, 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054648398000000001, 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1543 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221057+0000", 2024-09-14T07:08:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221058+0000", 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4299999999999999e-07 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221060+0000", 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7990000000000001e-06 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221066+0000", 2024-09-14T07:08:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221070+0000", 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7010000000000004e-06 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275609+0000", 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054538968 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275652+0000", 2024-09-14T07:08:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2864000000000002e-05 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275706+0000", 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3597000000000002e-05 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1545 3.8 3:17d01ef9:::rbd_data.116167e24a2d.0000000000000006:a [sparse-read 16646144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242103634999999, 2024-09-14T07:08:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054855590000000003, 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1545 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221077+0000", 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4499999999999998e-07 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221085+0000", 2024-09-14T07:08:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0330000000000008e-06 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221092+0000", 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.793e-06 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221098+0000", 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3090000000000003e-06 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275744+0000", 2024-09-14T07:08:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054646404000000003 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275821+0000", 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6793999999999997e-05 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275933+0000", 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111912 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1583 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 524288~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242075472000003, 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055023028000000002, 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1583 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221105+0000", 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221106+0000", 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221108+0000", 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0229999999999999e-06 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221121+0000", 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3645000000000001e-05 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221127+0000", 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.541e-06 2024-09-14T07:08:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275735+0000", 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054608372000000002 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.275817+0000", 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2083999999999995e-05 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276128+0000", 2024-09-14T07:08:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031090000000000002 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1585 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1114112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242045638, 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055153874999999998, 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1585 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221135+0000", 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221136+0000", 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7300000000000001e-07 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221138+0000", 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-09-14T07:08:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221144+0000", 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5100000000000004e-06 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221149+0000", 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.843e-06 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276165+0000", 2024-09-14T07:08:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055015677999999998 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276231+0000", 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5959999999999999e-05 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276289+0000", 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8100000000000003e-05 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1586 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.242022024000001, 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055262004000000003, 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1586 2024-09-14T07:08:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221166+0000", 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3180000000000001e-06 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221173+0000", 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221178+0000", 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2050000000000001e-06 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276321+0000", 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055142663000000001 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276364+0000", 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3615000000000002e-05 2024-09-14T07:08:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276421+0000", 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6453000000000003e-05 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1588 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.241994787000003, 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055364653999999999, 2024-09-14T07:08:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1588 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221189+0000", 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3269999999999999e-06 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221195+0000", 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2169999999999999e-06 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221200+0000", 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276451+0000", 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055251452999999999 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276497+0000", 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5114000000000002e-05 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276551+0000", 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4079999999999997e-05 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1589 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.241974122999999, 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055473737000000002, 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1589 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221209+0000", 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9269999999999999e-06 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221215+0000", 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3069999999999999e-06 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221220+0000", 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6369999999999998e-06 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276582+0000", 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055361856000000001 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276625+0000", 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2803000000000003e-05 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276680+0000", 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5789999999999999e-05 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1591 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.241953762999998, 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055584770999999998, 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1591 2024-09-14T07:08:53.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222763+0000", 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001535914 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222789+0000", 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6194e-05 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222862+0000", 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2529999999999998e-05 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222953+0000", 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1185000000000002e-05 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276713+0000", 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053760371000000001 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276758+0000", 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4932000000000003e-05 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276812+0000", 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3644999999999999e-05 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1593 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 6225920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.240183088000002, 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054086803000000003, 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1593 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222998+0000", 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223000+0000", 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.289e-06 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223007+0000", 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.058e-06 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223013+0000", 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5779999999999998e-06 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276973+0000", 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053960263000000001 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277028+0000", 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000002e-05 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277084+0000", 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6357999999999998e-05 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1594 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.240159222999999, 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054193419, 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1594 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223022+0000", 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223035+0000", 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2994e-05 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223043+0000", 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6539999999999999e-06 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223048+0000", 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6149999999999996e-06 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277114+0000", 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054066256 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277157+0000", 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2629000000000001e-05 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277215+0000", 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7803999999999997e-05 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1598 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.239539667999999, 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054092236000000002, 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1598 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.223641+0000", 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224174+0000", 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000533265 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224179+0000", 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2420000000000002e-06 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224190+0000", 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-05 2024-09-14T07:08:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.224197+0000", 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3549999999999999e-06 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277635+0000", 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053437314999999999 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277679+0000", 2024-09-14T07:08:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4628e-05 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.277733+0000", 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4131000000000001e-05 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.162874031999998, 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:08:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:08:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:08:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.117883726000002, 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:08:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.815951269999999, 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:08:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:08:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:08:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.494623535999999, 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:08:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:53.558 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:53.559 INFO:teuthology.orchestra.run.smithi145.stdout:1988 trunc from 0xce11600 to 0x878ec00 2024-09-14T07:08:53.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:53.560 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:53.557+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:53.660 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:53.657+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:53.668 INFO:teuthology.orchestra.run.smithi145.stdout:1989 read 0x5e54a00 thru 0x5e5523c (0x83d bytes) 2024-09-14T07:08:53.671 INFO:teuthology.orchestra.run.smithi145.stdout:1991 punch from 0x37f2400 to 0x37fae00, (0x8a00 bytes) 2024-09-14T07:08:53.709 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:53.709 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:53.709 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:53.709 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:53.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:53.744 INFO:teuthology.orchestra.run.smithi145.stdout:1992 write 0xe60a400 thru 0xe619078 (0xec79 bytes) 2024-09-14T07:08:53.745 INFO:teuthology.orchestra.run.smithi145.stdout:1993 write 0x9a9d000 thru 0x9aac37a (0xf37b bytes) 2024-09-14T07:08:53.745 INFO:teuthology.orchestra.run.smithi145.stdout:1995 read 0xcec5400 thru 0xcec5fa7 (0xba8 bytes) 2024-09-14T07:08:53.761 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:53.757+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:53.798 INFO:teuthology.orchestra.run.smithi145.stdout:1996 trunc from 0xe619200 to 0x5b60a00 2024-09-14T07:08:53.863 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:53.861+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:53.915 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:53.915 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:53.932 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:53.975 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:53.973+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:54.060 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.758881650000006, 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:54.061 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:54.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:54.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:54.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:54.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1503 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.838442477999997, 2024-09-14T07:08:54.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042296750000000001, 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1503 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220943+0000", 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220945+0000", 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.013e-06 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220951+0000", 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.806e-06 2024-09-14T07:08:54.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220970+0000", 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8649999999999999e-05 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.220983+0000", 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3964999999999999e-05 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.262989+0000", 2024-09-14T07:08:54.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042005845 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263116+0000", 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012688 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263240+0000", 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012359099999999999 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1504 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.838383354000001, 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042493942999999999, 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1504 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221002+0000", 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221004+0000", 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.063e-06 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221011+0000", 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8059999999999997e-06 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221030+0000", 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8652999999999999e-05 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221044+0000", 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3879999999999999e-05 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263275+0000", 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042231869999999998 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263365+0000", 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9813000000000002e-05 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.263496+0000", 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000130858 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1636 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.834655667, 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042143574000000003, 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1636 2024-09-14T07:08:54.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224730+0000", 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224731+0000", 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5600000000000001e-06 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224738+0000", 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224756+0000", 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8153999999999999e-05 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224770+0000", 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4168999999999999e-05 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266618+0000", 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041847662000000001 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266730+0000", 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000111993 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266873+0000", 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014374400000000001 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1644 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 9568256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833516404000001, 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042307838, 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.080 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1644 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225869+0000", 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225870+0000", 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2270000000000001e-06 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225876+0000", 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2569999999999998e-06 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225894+0000", 2024-09-14T07:08:54.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8385000000000001e-05 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225936+0000", 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2159000000000001e-05 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.267952+0000", 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04201585 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268067+0000", 2024-09-14T07:08:54.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114985 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268177+0000", 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109975 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1645 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 10158080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833428026999997, 2024-09-14T07:08:54.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042484558999999998, 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1645 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225957+0000", 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225959+0000", 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225978+0000", 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9466e-05 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225998+0000", 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9315999999999999e-05 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226013+0000", 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4919999999999999e-05 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268213+0000", 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200410000000001 2024-09-14T07:08:54.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268324+0000", 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011123799999999999 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268442+0000", 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117889 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.088 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1646 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12320768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833351649000001, 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042595283999999997, 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1646 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226034+0000", 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226035+0000", 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5060000000000001e-06 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226042+0000", 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4370000000000004e-06 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226060+0000", 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8266000000000001e-05 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226074+0000", 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4029000000000001e-05 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042405023999999999 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268551+0000", 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1973999999999999e-05 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268629+0000", 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8047999999999995e-05 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1647 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 12910592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833293615000002, 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042800796000000002, 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1647 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226092+0000", 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226093+0000", 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.46e-06 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226099+0000", 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3530000000000002e-06 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226118+0000", 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9055999999999998e-05 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226132+0000", 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3813e-05 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268673+0000", 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042541272999999998 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268775+0000", 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102302 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268893+0000", 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000117539 2024-09-14T07:08:54.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1648 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 13500416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833234474000001, 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042904468000000001, 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1648 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226151+0000", 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226152+0000", 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.234e-06 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226157+0000", 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2000000000000002e-06 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226175+0000", 2024-09-14T07:08:54.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7059e-05 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226204+0000", 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9536e-05 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268938+0000", 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042733698000000001 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268992+0000", 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4693e-05 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269056+0000", 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3047999999999997e-05 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1649 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 14090240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.101 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833160266, 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043088794999999999, 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1649 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226225+0000", 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226227+0000", 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.33e-06 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226232+0000", 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.733e-06 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226251+0000", 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8816000000000001e-05 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226264+0000", 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-05 2024-09-14T07:08:54.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269097+0000", 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0428324 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269197+0000", 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100419 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269314+0000", 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000116897 2024-09-14T07:08:54.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1651 3.13 3:cbed45ec:::rbd_data.116167e24a2d.0000000000000001:a [sparse-read 16252928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833101417999998, 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043199128000000003, 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1651 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226284+0000", 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226285+0000", 2024-09-14T07:08:54.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.237e-06 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226291+0000", 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1789999999999999e-06 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226311+0000", 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0656999999999999e-05 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226327+0000", 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5948999999999999e-05 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269352+0000", 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043024830999999999 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269419+0000", 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7281999999999997e-05 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269483+0000", 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3992999999999995e-05 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1700 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 32768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.833038090000002, 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043417208999999998, 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1700 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226347+0000", 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226349+0000", 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3629999999999999e-06 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226354+0000", 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226372+0000", 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7612999999999999e-05 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226386+0000", 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4146e-05 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269525+0000", 2024-09-14T07:08:54.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043139380999999997 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269631+0000", 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105574 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269765+0000", 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133759 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1702 3.13 3:cfeabdd5:::rbd_data.11616dafc67f.000000000000000f:9 [read 491520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.832949669999998, 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043560261000000003, 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1702 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226436+0000", 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226437+0000", 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.567e-06 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226457+0000", 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395999999999999e-05 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226477+0000", 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9998999999999999e-05 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226491+0000", 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4503e-05 2024-09-14T07:08:54.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269835+0000", 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043343995000000003 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269914+0000", 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9018999999999997e-05 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269996+0000", 2024-09-14T07:08:54.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1781999999999997e-05 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1740 3.13 3:cda6db13:::rbd_data.11616dafc67f.0000000000000009:9 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.832872776000002, 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043673828999999997, 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.119 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1740 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226513+0000", 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226514+0000", 2024-09-14T07:08:54.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4869999999999999e-06 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226520+0000", 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.976e-06 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226538+0000", 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7972999999999999e-05 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.226551+0000", 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3142999999999999e-05 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270041+0000", 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043490174999999999 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270101+0000", 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9515999999999997e-05 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270187+0000", 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5558999999999999e-05 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.768098514999998, 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:54.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:54.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4803.0:1903 3.1f e43/21 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head v 43'21, mlcod=42'19)", 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.750937696000001, 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.057064131999999997, 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:54.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308449+0000", 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.279e-06 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324156+0000", 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015707199000000002 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324195+0000", 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9070999999999999e-05 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324217+0000", 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1752000000000001e-05 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324333+0000", 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011607000000000001 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324359+0000", 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6296e-05 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365477+0000", 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041117065000000001 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365512+0000", 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-05 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4703.0:530 3.f e46/21 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head v 46'75, mlcod=45'73)", 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.196998751000002, 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.075870370000000006, 2024-09-14T07:08:54.133 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862389+0000", 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6579999999999998e-06 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870883+0000", 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084939100000000003 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870916+0000", 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2428000000000002e-05 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870931+0000", 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5656000000000001e-05 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871022+0000", 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1063000000000005e-05 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871047+0000", 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4125e-05 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938212+0000", 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067164920000000003 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938257+0000", 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5609999999999999e-05 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.137 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.864888466, 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:08:54.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:54.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.145 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.8647685870000004, 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:08:54.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:08:54.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:08:54.153 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.8475434670000004, 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:54.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:08:54.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:08:54.158 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.158 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.158 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.158 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:54.158 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:54.158 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:54.160 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:54.160 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:54.157+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:54.217 INFO:teuthology.orchestra.run.smithi145.stdout:1997 trunc from 0x5b60a00 to 0x96a5e00 2024-09-14T07:08:54.238 INFO:teuthology.orchestra.run.smithi145.stdout:1998 punch from 0x62c3200 to 0x62c9200, (0x6000 bytes) 2024-09-14T07:08:54.239 INFO:teuthology.orchestra.run.smithi145.stdout:1999 punch from 0x7d83800 to 0x7d87c00, (0x4400 bytes) 2024-09-14T07:08:54.239 INFO:teuthology.orchestra.run.smithi145.stdout:2000 write 0x295c000 thru 0x2965b0a (0x9b0b bytes) 2024-09-14T07:08:54.240 INFO:teuthology.orchestra.run.smithi145.stdout:2004 read 0x150400 thru 0x158d8d (0x898e bytes) 2024-09-14T07:08:54.244 INFO:teuthology.orchestra.run.smithi145.stdout:2005 read 0x80b8e00 thru 0x80c3976 (0xab77 bytes) 2024-09-14T07:08:54.248 INFO:teuthology.orchestra.run.smithi145.stdout:2007 trunc from 0x96a5e00 to 0xaa9e200 2024-09-14T07:08:54.260 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:54.257+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:54.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:54.313 INFO:teuthology.orchestra.run.smithi145.stdout:2009 read 0x421fa00 thru 0x4225631 (0x5c32 bytes) 2024-09-14T07:08:54.339 INFO:teuthology.orchestra.run.smithi145.stdout:2010 write 0xcfaa800 thru 0xcfb81ce (0xd9cf bytes) 2024-09-14T07:08:54.340 INFO:teuthology.orchestra.run.smithi145.stdout:2012 write 0xb534400 thru 0xb53d152 (0x8d53 bytes) 2024-09-14T07:08:54.340 INFO:teuthology.orchestra.run.smithi145.stdout:2013 write 0x7a07800 thru 0x7a0c1d6 (0x49d7 bytes) 2024-09-14T07:08:54.340 INFO:teuthology.orchestra.run.smithi145.stdout:2014 write 0x12c6200 thru 0x12c7933 (0x1734 bytes) 2024-09-14T07:08:54.341 INFO:teuthology.orchestra.run.smithi145.stdout:2018 read 0x653c00 thru 0x65e4ca (0xa8cb bytes) 2024-09-14T07:08:54.346 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:54.346 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:54.346 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:54.346 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:54.348 INFO:teuthology.orchestra.run.smithi145.stdout:2022 trunc from 0xcfb8200 to 0xe706e00 2024-09-14T07:08:54.361 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:54.357+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:54.363 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:54.455 INFO:teuthology.orchestra.run.smithi145.stdout:2024 trunc from 0xe706e00 to 0x1116000 2024-09-14T07:08:54.462 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:54.457+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:54.503 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:54.544 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:54.563 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:54.563 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:54.563 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:54.563 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:54.563 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:54.567 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:54.565+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:54.570 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:54.570 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:54.584 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:54.589 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:54.609 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:54.630 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:54.668 INFO:teuthology.orchestra.run.smithi145.stdout:2025 trunc from 0x1116000 to 0x6848000 2024-09-14T07:08:54.671 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:54.669+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:54.746 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:54.747 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:54.761 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:54.804 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:54.801+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:54.866 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:54.861+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:54.928 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:54.928 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:54.942 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:54.964 INFO:teuthology.orchestra.run.smithi145.stdout:2027 write 0x9827200 thru 0x982c2ea (0x50eb bytes) 2024-09-14T07:08:54.967 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:54.965+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:54.967 INFO:teuthology.orchestra.run.smithi145.stdout:2028 write 0x455f400 thru 0x456ae1c (0xba1d bytes) 2024-09-14T07:08:54.968 INFO:teuthology.orchestra.run.smithi145.stdout:2032 write 0x212ee00 thru 0x213d2bb (0xe4bc bytes) 2024-09-14T07:08:54.968 INFO:teuthology.orchestra.run.smithi145.stdout:2033 write 0x45fda00 thru 0x45ff98f (0x1f90 bytes) 2024-09-14T07:08:55.006 INFO:teuthology.orchestra.run.smithi145.stdout:2034 trunc from 0x982c400 to 0xd3200 2024-09-14T07:08:55.068 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:55.065+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:55.079 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:55.079 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:55.094 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:55.105 INFO:teuthology.orchestra.run.smithi145.stdout:2035 trunc from 0xd3200 to 0x4109c00 2024-09-14T07:08:55.165 INFO:teuthology.orchestra.run.smithi145.stdout:2036 write 0x830ca00 thru 0x830e39d (0x199e bytes) 2024-09-14T07:08:55.166 INFO:teuthology.orchestra.run.smithi145.stdout:2037 read 0x528a600 thru 0x52930a0 (0x8aa1 bytes) 2024-09-14T07:08:55.172 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:55.165+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:55.197 INFO:teuthology.orchestra.run.smithi145.stdout:2038 write 0x8a34a00 thru 0x8a3ca28 (0x8029 bytes) 2024-09-14T07:08:55.204 INFO:teuthology.orchestra.run.smithi145.stdout:2039 trunc from 0x8a3cc00 to 0xb2f1000 2024-09-14T07:08:55.241 INFO:teuthology.orchestra.run.smithi145.stdout:2040 write 0xa86a400 thru 0xa8788bb (0xe4bc bytes) 2024-09-14T07:08:55.243 INFO:teuthology.orchestra.run.smithi145.stdout:2041 flatten 2024-09-14T07:08:55.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:08:55.273 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:55.273 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:55.284 INFO:teuthology.orchestra.run.smithi145.stdout:2042 trunc from 0xb2f1000 to 0xc856400 2024-09-14T07:08:55.288 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:55.301 INFO:teuthology.orchestra.run.smithi145.stdout:2044 punch from 0x6c96a00 to 0x6ca3a00, (0xd000 bytes) 2024-09-14T07:08:55.303 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:55.301+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:55.304 INFO:teuthology.orchestra.run.smithi145.stdout:2046 punch from 0x82cf000 to 0x82cf400, (0x400 bytes) 2024-09-14T07:08:55.304 INFO:teuthology.orchestra.run.smithi145.stdout:truncating to largest ever: 0xee55000 2024-09-14T07:08:55.315 INFO:teuthology.orchestra.run.smithi145.stdout:2047 trunc from 0xc856400 to 0xee55000 2024-09-14T07:08:55.333 INFO:teuthology.orchestra.run.smithi145.stdout:2049 trunc from 0xee55000 to 0xda0ee00 2024-09-14T07:08:55.349 INFO:teuthology.orchestra.run.smithi145.stdout:2050 read 0x9110c00 thru 0x911349e (0x289f bytes) 2024-09-14T07:08:55.352 INFO:teuthology.orchestra.run.smithi145.stdout:2052 punch from 0xcb7a000 to 0xcb7ea00, (0x4a00 bytes) 2024-09-14T07:08:55.352 INFO:teuthology.orchestra.run.smithi145.stdout:2054 read 0xda6000 thru 0xdabdcd (0x5dce bytes) 2024-09-14T07:08:55.353 INFO:teuthology.orchestra.run.smithi145.stdout:2055 punch from 0x10de200 to 0x10e5800, (0x7600 bytes) 2024-09-14T07:08:55.353 INFO:teuthology.orchestra.run.smithi145.stdout:2056 punch from 0x294f400 to 0x295ea00, (0xf600 bytes) 2024-09-14T07:08:55.354 INFO:teuthology.orchestra.run.smithi145.stdout:2058 read 0x1b2f800 thru 0x1b34400 (0x4c01 bytes) 2024-09-14T07:08:55.358 INFO:teuthology.orchestra.run.smithi145.stdout:2059 punch from 0x43a1c00 to 0x43a8200, (0x6600 bytes) 2024-09-14T07:08:55.359 INFO:teuthology.orchestra.run.smithi145.stdout:2061 read 0xc88b600 thru 0xc88cfab (0x19ac bytes) 2024-09-14T07:08:55.359 INFO:teuthology.orchestra.run.smithi145.stdout:2062 punch from 0x8921000 to 0x892e000, (0xd000 bytes) 2024-09-14T07:08:55.359 INFO:teuthology.orchestra.run.smithi145.stdout:2065 punch from 0xd16c600 to 0xd171200, (0x4c00 bytes) 2024-09-14T07:08:55.359 INFO:teuthology.orchestra.run.smithi145.stdout:2066 write 0xa08e800 thru 0xa099ef8 (0xb6f9 bytes) 2024-09-14T07:08:55.360 INFO:teuthology.orchestra.run.smithi145.stdout:2067 read 0x842aa00 thru 0x84308e9 (0x5eea bytes) 2024-09-14T07:08:55.360 INFO:teuthology.orchestra.run.smithi145.stdout:2068 read 0x85f6800 thru 0x8600115 (0x9916 bytes) 2024-09-14T07:08:55.360 INFO:teuthology.orchestra.run.smithi145.stdout:2071 read 0xa854200 thru 0xa8565f2 (0x23f3 bytes) 2024-09-14T07:08:55.372 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:55.369+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:55.380 INFO:teuthology.orchestra.run.smithi145.stdout:2072 trunc from 0xda0ee00 to 0x146ae00 2024-09-14T07:08:55.454 INFO:teuthology.orchestra.run.smithi145.stdout:2073 trunc from 0x146ae00 to 0x7529400 2024-09-14T07:08:55.467 INFO:teuthology.orchestra.run.smithi145.stdout:2074 read 0x6a9d400 thru 0x6aa2b6c (0x576d bytes) 2024-09-14T07:08:55.468 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:55.468 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:55.469 INFO:teuthology.orchestra.run.smithi145.stdout:2076 read 0x5c12800 thru 0x5c219a7 (0xf1a8 bytes) 2024-09-14T07:08:55.472 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:55.469+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:55.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:55.495 INFO:teuthology.orchestra.run.smithi145.stdout:2077 write 0xc687000 thru 0xc68c1ff (0x5200 bytes) 2024-09-14T07:08:55.495 INFO:teuthology.orchestra.run.smithi145.stdout:2078 read 0x4c3fa00 thru 0x4c491d7 (0x97d8 bytes) 2024-09-14T07:08:55.497 INFO:teuthology.orchestra.run.smithi145.stdout:2079 read 0xbf37a00 thru 0xbf43fea (0xc5eb bytes) 2024-09-14T07:08:55.497 INFO:teuthology.orchestra.run.smithi145.stdout:2080 write 0x3de0400 thru 0x3de4e30 (0x4a31 bytes) 2024-09-14T07:08:55.514 INFO:teuthology.orchestra.run.smithi145.stdout:2081 trunc from 0xc68c200 to 0x20a9800 2024-09-14T07:08:55.545 INFO:teuthology.orchestra.run.smithi145.stdout:2084 punch from 0x2be200 to 0x2c0800, (0x2600 bytes) 2024-09-14T07:08:55.564 INFO:teuthology.orchestra.run.smithi145.stdout:2088 write 0x2383200 thru 0x2390140 (0xcf41 bytes) 2024-09-14T07:08:55.566 INFO:teuthology.orchestra.run.smithi145.stdout:2089 punch from 0x1087600 to 0x1088600, (0x1000 bytes) 2024-09-14T07:08:55.567 INFO:teuthology.orchestra.run.smithi145.stdout:2091 read 0x1bbee00 thru 0x1bce203 (0xf404 bytes) 2024-09-14T07:08:55.574 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:55.569+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:55.591 INFO:teuthology.orchestra.run.smithi145.stdout:2099 write 0x2f42600 thru 0x2f44b8c (0x258d bytes) 2024-09-14T07:08:55.619 INFO:teuthology.orchestra.run.smithi145.stdout:2100 write 0x603ae00 thru 0x603d79f (0x29a0 bytes) 2024-09-14T07:08:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:55.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:55.660 INFO:teuthology.orchestra.run.smithi145.stdout:2101 write 0xbf9bc00 thru 0xbfa9837 (0xdc38 bytes) 2024-09-14T07:08:55.663 INFO:teuthology.orchestra.run.smithi145.stdout:2102 write 0xb309600 thru 0xb30b986 (0x2387 bytes) 2024-09-14T07:08:55.663 INFO:teuthology.orchestra.run.smithi145.stdout:2104 read 0x4300400 thru 0x430efee (0xebef bytes) 2024-09-14T07:08:55.683 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:55.681+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:55.683 INFO:teuthology.orchestra.run.smithi145.stdout:2115 trunc from 0xbfa9a00 to 0x270aa00 2024-09-14T07:08:55.723 INFO:teuthology.orchestra.run.smithi145.stdout:2117 trunc from 0x270aa00 to 0x27da600 2024-09-14T07:08:55.747 INFO:teuthology.orchestra.run.smithi145.stdout:2118 trunc from 0x27da600 to 0x4172800 2024-09-14T07:08:55.762 INFO:teuthology.orchestra.run.smithi145.stdout:2120 punch from 0x5ada00 to 0x5b0e00, (0x3400 bytes) 2024-09-14T07:08:55.763 INFO:teuthology.orchestra.run.smithi145.stdout:2121 read 0x35f2000 thru 0x35fae27 (0x8e28 bytes) 2024-09-14T07:08:55.775 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:55.769+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:55.801 INFO:teuthology.orchestra.run.smithi145.stdout:2123 write 0xbb28800 thru 0xbb2d1da (0x49db bytes) 2024-09-14T07:08:55.802 INFO:teuthology.orchestra.run.smithi145.stdout:2124 write 0x48df200 thru 0x48e0f2c (0x1d2d bytes) 2024-09-14T07:08:55.803 INFO:teuthology.orchestra.run.smithi145.stdout:2126 write 0x7820800 thru 0x782ef66 (0xe767 bytes) 2024-09-14T07:08:55.803 INFO:teuthology.orchestra.run.smithi145.stdout:2128 read 0x50ea800 thru 0x50f9291 (0xea92 bytes) 2024-09-14T07:08:55.818 INFO:teuthology.orchestra.run.smithi145.stdout:2129 trunc from 0xbb2d200 to 0x2cff800 2024-09-14T07:08:55.820 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:55.820 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:55.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:55.883 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:55.881+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:55.884 INFO:teuthology.orchestra.run.smithi145.stdout:2130 write 0x5664c00 thru 0x56653b9 (0x7ba bytes) 2024-09-14T07:08:55.886 INFO:teuthology.orchestra.run.smithi145.stdout:2131 punch from 0x1a77e00 to 0x1a82400, (0xa600 bytes) 2024-09-14T07:08:55.886 INFO:teuthology.orchestra.run.smithi145.stdout:2132 read 0x35ae600 thru 0x35b364f (0x5050 bytes) 2024-09-14T07:08:55.887 INFO:teuthology.orchestra.run.smithi145.stdout:2133 write 0x1c71a00 thru 0x1c7daba (0xc0bb bytes) 2024-09-14T07:08:55.887 INFO:teuthology.orchestra.run.smithi145.stdout:2134 read 0x3c77000 thru 0x3c78558 (0x1559 bytes) 2024-09-14T07:08:55.887 INFO:teuthology.orchestra.run.smithi145.stdout:2135 read 0x1106800 thru 0x11135be (0xcdbf bytes) 2024-09-14T07:08:55.893 INFO:teuthology.orchestra.run.smithi145.stdout:2136 read 0x3d0c400 thru 0x3d17d37 (0xb938 bytes) 2024-09-14T07:08:55.898 INFO:teuthology.orchestra.run.smithi145.stdout:2137 trunc from 0x5665400 to 0x340b400 2024-09-14T07:08:55.934 INFO:teuthology.orchestra.run.smithi145.stdout:2140 punch from 0x28ec000 to 0x28f6000, (0xa000 bytes) 2024-09-14T07:08:55.937 INFO:teuthology.orchestra.run.smithi145.stdout:2141 punch from 0x2534c00 to 0x253a400, (0x5800 bytes) 2024-09-14T07:08:55.937 INFO:teuthology.orchestra.run.smithi145.stdout:2142 read 0xb08a00 thru 0xb1466b (0xbc6c bytes) 2024-09-14T07:08:55.963 INFO:teuthology.orchestra.run.smithi145.stdout:2143 write 0x8820400 thru 0x882e652 (0xe253 bytes) 2024-09-14T07:08:55.973 INFO:teuthology.orchestra.run.smithi145.stdout:2144 trunc from 0x882e800 to 0x58f8c00 2024-09-14T07:08:55.976 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:55.973+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:55.978 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:55.978 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:55.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:56.006 INFO:teuthology.orchestra.run.smithi145.stdout:2145 read 0x4f53400 thru 0x4f61d2c (0xe92d bytes) 2024-09-14T07:08:56.008 INFO:teuthology.orchestra.run.smithi145.stdout:2146 read 0x5783c00 thru 0x578ce6f (0x9270 bytes) 2024-09-14T07:08:56.008 INFO:teuthology.orchestra.run.smithi145.stdout:2148 punch from 0x42be000 to 0x42c4800, (0x6800 bytes) 2024-09-14T07:08:56.037 INFO:teuthology.orchestra.run.smithi145.stdout:2149 write 0xc26a200 thru 0xc26fe2f (0x5c30 bytes) 2024-09-14T07:08:56.040 INFO:teuthology.orchestra.run.smithi145.stdout:2151 write 0xb882e00 thru 0xb88d13e (0xa33f bytes) 2024-09-14T07:08:56.040 INFO:teuthology.orchestra.run.smithi145.stdout:2153 write 0x88cfe00 thru 0x88dc3ff (0xc600 bytes) 2024-09-14T07:08:56.040 INFO:teuthology.orchestra.run.smithi145.stdout:2155 write 0xac06e00 thru 0xac15513 (0xe714 bytes) 2024-09-14T07:08:56.040 INFO:teuthology.orchestra.run.smithi145.stdout:2156 read 0xb045800 thru 0xb054bc2 (0xf3c3 bytes) 2024-09-14T07:08:56.041 INFO:teuthology.orchestra.run.smithi145.stdout:2157 read 0x41d600 thru 0x4256fe (0x80ff bytes) 2024-09-14T07:08:56.041 INFO:teuthology.orchestra.run.smithi145.stdout:2158 write 0xa92a00 thru 0xaa1421 (0xea22 bytes) 2024-09-14T07:08:56.050 INFO:teuthology.orchestra.run.smithi145.stdout:2161 trunc from 0xc270000 to 0x8000200 2024-09-14T07:08:56.077 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:56.073+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:56.106 INFO:teuthology.orchestra.run.smithi145.stdout:2162 write 0x2343c00 thru 0x2344997 (0xd98 bytes) 2024-09-14T07:08:56.109 INFO:teuthology.orchestra.run.smithi145.stdout:2163 write 0x6e77800 thru 0x6e7f18a (0x798b bytes) 2024-09-14T07:08:56.109 INFO:teuthology.orchestra.run.smithi145.stdout:2164 write 0x972c00 thru 0x97b828 (0x8c29 bytes) 2024-09-14T07:08:56.143 INFO:teuthology.orchestra.run.smithi145.stdout:2166 write 0x96e8800 thru 0x96f8180 (0xf981 bytes) 2024-09-14T07:08:56.144 INFO:teuthology.orchestra.run.smithi145.stdout:2167 write 0x2a0e000 thru 0x2a0e84b (0x84c bytes) 2024-09-14T07:08:56.145 INFO:teuthology.orchestra.run.smithi145.stdout:2170 read 0x3f29800 thru 0x3f2c73e (0x2f3f bytes) 2024-09-14T07:08:56.164 INFO:teuthology.orchestra.run.smithi145.stdout:2172 trunc from 0x96f8200 to 0x2615c00 2024-09-14T07:08:56.177 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:56.177 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:56.178 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:56.173+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:56.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:56.212 INFO:teuthology.orchestra.run.smithi145.stdout:2174 read 0x25ca200 thru 0x25cd796 (0x3597 bytes) 2024-09-14T07:08:56.230 INFO:teuthology.orchestra.run.smithi145.stdout:2177 trunc from 0x2615c00 to 0xb26da00 2024-09-14T07:08:56.249 INFO:teuthology.orchestra.run.smithi145.stdout:2178 read 0x855ea00 thru 0x85699ef (0xaff0 bytes) 2024-09-14T07:08:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:08:56.262 INFO:teuthology.orchestra.run.smithi145.stdout:2181 trunc from 0xb26da00 to 0x58f4a00 2024-09-14T07:08:56.278 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:56.273+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:56.319 INFO:teuthology.orchestra.run.smithi145.stdout:2183 write 0xe490e00 thru 0xe4a09be (0xfbbf bytes) 2024-09-14T07:08:56.321 INFO:teuthology.orchestra.run.smithi145.stdout:2185 read 0x6ef7200 thru 0x6f053f5 (0xe1f6 bytes) 2024-09-14T07:08:56.321 INFO:teuthology.orchestra.run.smithi145.stdout:2187 punch from 0x14eca00 to 0x14efc00, (0x3200 bytes) 2024-09-14T07:08:56.322 INFO:teuthology.orchestra.run.smithi145.stdout:2189 write 0x369aa00 thru 0x36a0353 (0x5954 bytes) 2024-09-14T07:08:56.322 INFO:teuthology.orchestra.run.smithi145.stdout:2190 punch from 0x6864400 to 0x686f600, (0xb200 bytes) 2024-09-14T07:08:56.323 INFO:teuthology.orchestra.run.smithi145.stdout:2191 write 0xc6b0400 thru 0xc6be4fb (0xe0fc bytes) 2024-09-14T07:08:56.323 INFO:teuthology.orchestra.run.smithi145.stdout:2193 punch from 0xdce3400 to 0xdcf1000, (0xdc00 bytes) 2024-09-14T07:08:56.344 INFO:teuthology.orchestra.run.smithi145.stdout:2194 trunc from 0xe4a0a00 to 0xd3de00 2024-09-14T07:08:56.379 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:56.377+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:56.384 INFO:teuthology.orchestra.run.smithi145.stdout:2196 punch from 0x152e00 to 0x153a00, (0xc00 bytes) 2024-09-14T07:08:56.391 INFO:teuthology.orchestra.run.smithi145.stdout:2197 trunc from 0xd3de00 to 0x37d5600 2024-09-14T07:08:56.395 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:56.395 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:56.407 INFO:teuthology.orchestra.run.smithi145.stdout:2201 write 0x197aa00 thru 0x1988a8c (0xe08d bytes) 2024-09-14T07:08:56.408 INFO:teuthology.orchestra.run.smithi145.stdout:2202 write 0xbd8600 thru 0xbdda70 (0x5471 bytes) 2024-09-14T07:08:56.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:56.442 INFO:teuthology.orchestra.run.smithi145.stdout:2211 write 0x99e5400 thru 0x99e803b (0x2c3c bytes) 2024-09-14T07:08:56.444 INFO:teuthology.orchestra.run.smithi145.stdout:2212 read 0x807d000 thru 0x807f42b (0x242c bytes) 2024-09-14T07:08:56.444 INFO:teuthology.orchestra.run.smithi145.stdout:2214 write 0x693be00 thru 0x6941b95 (0x5d96 bytes) 2024-09-14T07:08:56.445 INFO:teuthology.orchestra.run.smithi145.stdout:2216 read 0x52ed400 thru 0x52fd180 (0xfd81 bytes) 2024-09-14T07:08:56.445 INFO:teuthology.orchestra.run.smithi145.stdout:2217 read 0x4b80a00 thru 0x4b85e60 (0x5461 bytes) 2024-09-14T07:08:56.445 INFO:teuthology.orchestra.run.smithi145.stdout:2218 read 0x3952e00 thru 0x395f48f (0xc690 bytes) 2024-09-14T07:08:56.481 INFO:teuthology.orchestra.run.smithi145.stdout:2220 write 0xe3ed600 thru 0xe3f9002 (0xba03 bytes) 2024-09-14T07:08:56.481 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:56.477+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:56.482 INFO:teuthology.orchestra.run.smithi145.stdout:2223 read 0x87e9a00 thru 0x87f8192 (0xe793 bytes) 2024-09-14T07:08:56.496 INFO:teuthology.orchestra.run.smithi145.stdout:2225 trunc from 0xe3f9200 to 0x703d200 2024-09-14T07:08:56.528 INFO:teuthology.orchestra.run.smithi145.stdout:2226 punch from 0x6218c00 to 0x6220200, (0x7600 bytes) 2024-09-14T07:08:56.531 INFO:teuthology.orchestra.run.smithi145.stdout:2228 trunc from 0x703d200 to 0x7c30a00 2024-09-14T07:08:56.582 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:56.577+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:56.588 INFO:teuthology.orchestra.run.smithi145.stdout:2231 write 0xb0daa00 thru 0xb0e6f1d (0xc51e bytes) 2024-09-14T07:08:56.600 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:56.601 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:56.602 INFO:teuthology.orchestra.run.smithi145.stdout:2232 trunc from 0xb0e7000 to 0x61bf400 2024-09-14T07:08:56.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:08:56.642 INFO:teuthology.orchestra.run.smithi145.stdout:2233 punch from 0x19b8600 to 0x19c0c00, (0x8600 bytes) 2024-09-14T07:08:56.644 INFO:teuthology.orchestra.run.smithi145.stdout:2234 read 0x3ed5a00 thru 0x3ee0cf4 (0xb2f5 bytes) 2024-09-14T07:08:56.645 INFO:teuthology.orchestra.run.smithi145.stdout:2236 read 0x3817200 thru 0x38263fd (0xf1fe bytes) 2024-09-14T07:08:56.645 INFO:teuthology.orchestra.run.smithi145.stdout:2237 read 0x139d200 thru 0x139df4f (0xd50 bytes) 2024-09-14T07:08:56.648 INFO:teuthology.orchestra.run.smithi145.stdout:2242 read 0x3814200 thru 0x381e123 (0x9f24 bytes) 2024-09-14T07:08:56.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:08:56.668 INFO:teuthology.orchestra.run.smithi145.stdout:2243 write 0x903de00 thru 0x9049f10 (0xc111 bytes) 2024-09-14T07:08:56.670 INFO:teuthology.orchestra.run.smithi145.stdout:2244 clone 15 order 18 su 131072 sc 9 2024-09-14T07:08:56.711 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:56.709+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:56.759 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:56.759 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:56.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:08:56.784 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:56.781+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:56.804 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:56.821 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:56.838 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:56.880 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:56.885 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:56.881+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:56.901 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:56.902 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:56.922 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:56.922 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1586 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 1703936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 39.700080192000001, 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055262004000000003, 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1586 2024-09-14T07:08:56.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221159+0000", 2024-09-14T07:08:56.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221166+0000", 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3180000000000001e-06 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221173+0000", 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221178+0000", 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2050000000000001e-06 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276321+0000", 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055142663000000001 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276364+0000", 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3615000000000002e-05 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276421+0000", 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6453000000000003e-05 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1588 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 2293760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 39.700052954999997, 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055364653999999999, 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:56.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1588 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221186+0000", 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221189+0000", 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3269999999999999e-06 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221195+0000", 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2169999999999999e-06 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221200+0000", 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276451+0000", 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055251452999999999 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276497+0000", 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5114000000000002e-05 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276551+0000", 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4079999999999997e-05 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1589 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 39.700032290999999, 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055473737000000002, 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1589 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221207+0000", 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221209+0000", 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9269999999999999e-06 2024-09-14T07:08:56.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221215+0000", 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3069999999999999e-06 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221220+0000", 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6369999999999998e-06 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276582+0000", 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055361856000000001 2024-09-14T07:08:56.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276625+0000", 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2803000000000003e-05 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276680+0000", 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5789999999999999e-05 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1591 3.18 3:1ff24e4d:::rbd_data.116167e24a2d.000000000000000c:a [sparse-read 5046272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 39.700011930999999, 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055584770999999998, 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1591 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.936 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.221227+0000", 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222763+0000", 2024-09-14T07:08:56.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001535914 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222789+0000", 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6194e-05 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222862+0000", 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2529999999999998e-05 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.222953+0000", 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1185000000000002e-05 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276713+0000", 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053760371000000001 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276758+0000", 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4932000000000003e-05 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.276812+0000", 2024-09-14T07:08:56.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3644999999999999e-05 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 39.620932199999999, 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:08:56.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:08:56.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 39.575941894000003, 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.946 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:08:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:08:56.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:08:56.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:08:56.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 35.274009438, 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:08:56.951 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:56.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 34.952681704, 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:08:56.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4046 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 17088512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100340+0000", 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8208990009999999, 2024-09-14T07:08:56.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055368016999999999, 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4046 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100340+0000", 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100340+0000", 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100341+0000", 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4470000000000001e-06 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100346+0000", 2024-09-14T07:08:56.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0830000000000003e-06 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100363+0000", 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6586e-05 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100376+0000", 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3030000000000001e-05 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.155254+0000", 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.054878188000000001 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.155473+0000", 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000218357 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.155708+0000", 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000235326 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4047 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 17383424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100394+0000", 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8208453090000001, 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055595131999999998, 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4047 2024-09-14T07:08:56.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100394+0000", 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100394+0000", 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100395+0000", 2024-09-14T07:08:56.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.124e-06 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100400+0000", 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6160000000000001e-06 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100419+0000", 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8777e-05 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100433+0000", 2024-09-14T07:08:56.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3716e-05 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.155774+0000", 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055341380000000003 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.155867+0000", 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2552000000000004e-05 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.155989+0000", 2024-09-14T07:08:56.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012196700000000001 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4048 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 17678336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100452+0000", 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8207872200000002, 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055988398000000002, 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4048 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100452+0000", 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100452+0000", 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100453+0000", 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3570000000000001e-06 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100459+0000", 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4999999999999999e-06 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100476+0000", 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6983e-05 2024-09-14T07:08:56.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100489+0000", 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156052+0000", 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055563083999999999 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156289+0000", 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023703 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156440+0000", 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015104199999999999 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4049 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 17973248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100533+0000", 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8207056580000001, 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.056156022, 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4049 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100533+0000", 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100533+0000", 2024-09-14T07:08:56.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100535+0000", 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.499e-06 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100541+0000", 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0739999999999998e-06 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100561+0000", 2024-09-14T07:08:56.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0659999999999999e-05 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100576+0000", 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4566000000000001e-05 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156505+0000", 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055929143000000001 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156589+0000", 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4138e-05 2024-09-14T07:08:56.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156689+0000", 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9942e-05 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4050 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 18268160~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100596+0000", 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8206425899999998, 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.056481205, 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4050 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100596+0000", 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100596+0000", 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100598+0000", 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100603+0000", 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4260000000000002e-06 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100625+0000", 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1552999999999998e-05 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100638+0000", 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3305999999999999e-05 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156757+0000", 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.056118806 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.156904+0000", 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000147312 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157078+0000", 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017342799999999999 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4051 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 18563072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100655+0000", 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8205836440000001, 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.056756099999999997, 2024-09-14T07:08:56.983 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4051 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100655+0000", 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100655+0000", 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100656+0000", 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7000000000000003e-07 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100661+0000", 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9860000000000002e-06 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100678+0000", 2024-09-14T07:08:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6702999999999999e-05 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100691+0000", 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.275e-05 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157149+0000", 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.056458622 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157280+0000", 2024-09-14T07:08:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013052699999999999 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157411+0000", 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013154200000000001 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4052 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 18628608~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100712+0000", 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8205273850000001, 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057054668000000003, 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4052 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100712+0000", 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100712+0000", 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100713+0000", 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7060000000000001e-06 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100719+0000", 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8730000000000004e-06 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100738+0000", 2024-09-14T07:08:56.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.906e-05 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100752+0000", 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3997e-05 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157478+0000", 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.056725483 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157605+0000", 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012717100000000001 2024-09-14T07:08:56.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157766+0000", 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000161378 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4053 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 18857984~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100775+0000", 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8204634070000001, 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057302000999999998, 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4053 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100775+0000", 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100775+0000", 2024-09-14T07:08:56.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100777+0000", 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.06e-06 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100782+0000", 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3770000000000004e-06 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100799+0000", 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.685e-05 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100816+0000", 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7189000000000001e-05 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157827+0000", 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057010950999999997 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.157950+0000", 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000122819 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.158077+0000", 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012775500000000001 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4054 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 18923520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100833+0000", 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8204058910000001, 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057686421000000002, 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4054 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100833+0000", 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100833+0000", 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100834+0000", 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.283e-06 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100839+0000", 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100856+0000", 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6623e-05 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100869+0000", 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3173000000000001e-05 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.158146+0000", 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057276272000000003 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.158356+0000", 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021005000000000001 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:56.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.158519+0000", 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016382699999999999 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4055 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 19218432~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.100937+0000", 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8203015200000001, 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058012867000000003, 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:56.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4055 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100937+0000", 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100937+0000", 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100939+0000", 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5460000000000001e-06 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100955+0000", 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5979999999999999e-05 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100978+0000", 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-05 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.100993+0000", 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5226e-05 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.158641+0000", 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057647974999999997 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.158789+0000", 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014802000000000001 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:57.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.158950+0000", 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016082 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4169 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 16154624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.105883+0000", 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8153559499999998, 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055439212000000002, 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:57.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4169 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105883+0000", 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105883+0000", 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:57.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105884+0000", 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8299999999999995e-07 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105889+0000", 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.04e-06 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105905+0000", 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6495999999999999e-05 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105918+0000", 2024-09-14T07:08:57.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2840000000000001e-05 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.161121+0000", 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055202438 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.161210+0000", 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8758000000000006e-05 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.161322+0000", 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112657 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4170 3.8 3:13e17647:::rbd_data.11613d73da60.0000000000000005:10 [read 16449536~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:54.105938+0000", 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.8153010119999999, 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055807900000000001, 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:08:57.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4170 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105938+0000", 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:08:57.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.105938+0000", 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.108453+0000", 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0025150519999999998 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.108467+0000", 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3933e-05 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.108495+0000", 2024-09-14T07:08:57.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8212999999999999e-05 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.108515+0000", 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9922999999999999e-05 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.161398+0000", 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.052883459000000001 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.161544+0000", 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000145859 2024-09-14T07:08:57.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:54.161746+0000", 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201461 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:08:57.011 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:57.013 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:08:57.013 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:57.009+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:57.113 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:57.109+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:57.145 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:57.145 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:57.145 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:57.145 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:57.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:08:57.214 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:57.213+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:57.315 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:57.313+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:57.339 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:57.339 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:57.355 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:08:57.416 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:57.413+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:57.479 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.176455700999995, 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.480 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:08:57.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:08:57.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.185672566000001, 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.492 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4803.0:1903 3.1f e43/21 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head v 43'21, mlcod=42'19)", 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.168511746999997, 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.057064131999999997, 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308448+0000", 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.308449+0000", 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.279e-06 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324156+0000", 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.015707199000000002 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324195+0000", 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9070999999999999e-05 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324217+0000", 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1752000000000001e-05 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324333+0000", 2024-09-14T07:08:57.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011607000000000001 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.324359+0000", 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6296e-05 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365477+0000", 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041117065000000001 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.365512+0000", 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.54e-05 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4703.0:530 3.f e46/21 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head v 46'75, mlcod=45'73)", 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.614572801999998, 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.075870370000000006, 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862389+0000", 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6579999999999998e-06 2024-09-14T07:08:57.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870883+0000", 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084939100000000003 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870916+0000", 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2428000000000002e-05 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870931+0000", 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5656000000000001e-05 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871022+0000", 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1063000000000005e-05 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871047+0000", 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4125e-05 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:57.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938212+0000", 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067164920000000003 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938257+0000", 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5609999999999999e-05 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:08:57.501 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2824625170000008, 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:08:57.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:08:57.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:08:57.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:08:57.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:08:57.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:08:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2823426379999994, 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:08:57.509 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:08:57.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:08:57.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:08:57.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:08:57.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2651175180000003, 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:08:57.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:08:57.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6619 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 1589248~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538078+0000", 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9388811869999998, 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047375498000000002, 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.522 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6619 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538078+0000", 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538078+0000", 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538079+0000", 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3500000000000002e-07 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538081+0000", 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4549999999999998e-06 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538089+0000", 2024-09-14T07:08:57.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3780000000000003e-06 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538095+0000", 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9089999999999997e-06 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585344+0000", 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047249323000000003 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585394+0000", 2024-09-14T07:08:57.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9923000000000003e-05 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585454+0000", 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0075000000000003e-05 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6620 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 1818624~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538104+0000", 2024-09-14T07:08:57.526 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9388560039999998, 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047517904, 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6620 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538104+0000", 2024-09-14T07:08:57.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538104+0000", 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538104+0000", 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7999999999999995e-07 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538107+0000", 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538114+0000", 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.469e-06 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538120+0000", 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8619999999999999e-06 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585494+0000", 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047373909999999998 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585538+0000", 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4381000000000001e-05 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585621+0000", 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3306999999999994e-05 2024-09-14T07:08:57.530 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6621 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 1884160~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538127+0000", 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.938832482, 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047627544000000001, 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6621 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538127+0000", 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538127+0000", 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538128+0000", 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5400000000000001e-07 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538130+0000", 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2029999999999999e-06 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538137+0000", 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.25e-06 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538143+0000", 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6350000000000003e-06 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585650+0000", 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047506869 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585693+0000", 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3055e-05 2024-09-14T07:08:57.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585755+0000", 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1977999999999995e-05 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6622 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 2179072~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538150+0000", 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.938809875, 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047743513000000001, 2024-09-14T07:08:57.535 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6622 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538150+0000", 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538150+0000", 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538150+0000", 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8199999999999999e-07 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538153+0000", 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2620000000000001e-06 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538160+0000", 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4200000000000001e-06 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538166+0000", 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8529999999999997e-06 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585783+0000", 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047617254999999997 2024-09-14T07:08:57.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585834+0000", 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1264000000000002e-05 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585893+0000", 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8777000000000003e-05 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6623 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 2473984~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538173+0000", 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.938786715, 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047863214000000001, 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6623 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.540 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538173+0000", 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538173+0000", 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538173+0000", 2024-09-14T07:08:57.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538176+0000", 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2510000000000001e-06 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538183+0000", 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3100000000000003e-06 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538189+0000", 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7529999999999998e-06 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585921+0000", 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047732044000000001 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.585971+0000", 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9922000000000001e-05 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586036+0000", 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5500999999999995e-05 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6624 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 2768896~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538196+0000", 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9387636759999998, 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048063908000000002, 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6624 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538196+0000", 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538196+0000", 2024-09-14T07:08:57.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538197+0000", 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4000000000000001e-07 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538199+0000", 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538205+0000", 2024-09-14T07:08:57.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538211+0000", 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586081+0000", 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047870010999999997 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586176+0000", 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4326000000000005e-05 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586260+0000", 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4225000000000001e-05 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6625 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 3063808~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.548 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538233+0000", 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9387269659999999, 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048204232999999999, 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6625 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538233+0000", 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538233+0000", 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538233+0000", 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4900000000000005e-07 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538240+0000", 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0910000000000004e-06 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538248+0000", 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8490000000000006e-06 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538254+0000", 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4009999999999996e-06 2024-09-14T07:08:57.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586300+0000", 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048045978000000003 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586348+0000", 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8171999999999999e-05 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586437+0000", 2024-09-14T07:08:57.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8993000000000007e-05 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6626 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 3358720~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538262+0000", 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9386977390000002, 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048435692000000002, 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6626 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538262+0000", 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538262+0000", 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538262+0000", 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3900000000000004e-07 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538265+0000", 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.368e-06 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538272+0000", 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8519999999999998e-06 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538277+0000", 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5940000000000001e-06 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586483+0000", 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048205308000000002 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586567+0000", 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4677000000000003e-05 2024-09-14T07:08:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586698+0000", 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000130254 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6627 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 3424256~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538287+0000", 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9386727979999998, 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048570962000000002, 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6627 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538287+0000", 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538287+0000", 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538287+0000", 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7800000000000002e-07 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538290+0000", 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5040000000000001e-06 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538297+0000", 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6559999999999992e-06 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538304+0000", 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1670000000000004e-06 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586717+0000", 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048412998999999998 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586770+0000", 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2936000000000003e-05 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586858+0000", 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8221999999999997e-05 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6628 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 3653632~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.561 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538311+0000", 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.938648395, 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048693688999999998, 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6628 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538311+0000", 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538311+0000", 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538312+0000", 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3700000000000001e-07 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538314+0000", 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2620000000000001e-06 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538321+0000", 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3159999999999999e-06 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538327+0000", 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6439999999999997e-06 2024-09-14T07:08:57.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586892+0000", 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048564677000000001 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.586943+0000", 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1691999999999999e-05 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587005+0000", 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1661000000000005e-05 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6629 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 3719168~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538334+0000", 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9386256409999998, 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048809919, 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6629 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538334+0000", 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538334+0000", 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538334+0000", 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538337+0000", 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2450000000000001e-06 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538345+0000", 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2689999999999999e-06 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538351+0000", 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8139999999999997e-06 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587034+0000", 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048682884000000003 2024-09-14T07:08:57.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587083+0000", 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9561000000000002e-05 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587144+0000", 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.071e-05 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6630 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 4014080~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538360+0000", 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9385999699999998, 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048915610999999998, 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6630 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538360+0000", 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538360+0000", 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538360+0000", 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9500000000000002e-07 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538363+0000", 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2859999999999998e-06 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538370+0000", 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4329999999999998e-06 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538376+0000", 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7740000000000001e-06 2024-09-14T07:08:57.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587171+0000", 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048795560000000002 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587218+0000", 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6888999999999998e-05 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587275+0000", 2024-09-14T07:08:57.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6974e-05 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4916.0:6631 3.d 3:b181d8ef:::rbd_data.11613d73da60.0000000000000002:10 [read 4308992~16384] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:54.538383+0000", 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.9385764320000001, 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049026872999999999, 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:57.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4916", 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 6631 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538383+0000", 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538383+0000", 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538384+0000", 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.75e-07 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538386+0000", 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2199999999999999e-06 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538393+0000", 2024-09-14T07:08:57.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1999999999999997e-06 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.538399+0000", 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.711e-06 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587301+0000", 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048902563000000003 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:57.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587353+0000", 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2024999999999997e-05 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:54.587410+0000", 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6579000000000002e-05 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:57.579 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:57.581 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:08:57.581 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:57.577+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:57.682 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:57.677+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:57.709 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:57.709 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:57.709 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:57.709 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:57.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:08:57.783 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:57.781+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:57.884 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:57.881+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:57.894 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:57.894 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:57.894 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:57.894 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:57.894 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:57.908 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:08:57.915 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T07:08:57.984 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:57.981+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:58.017 INFO:teuthology.orchestra.run.smithi145.stdout:2245 trunc from 0x904a000 to 0x4a14000 2024-09-14T07:08:58.035 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:58.035 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:58.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1404 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.814771276999998, 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044358094000000001, 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.036 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1404 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216853+0000", 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216859+0000", 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2669999999999997e-06 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216877+0000", 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7866000000000001e-05 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216891+0000", 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4712000000000001e-05 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260962+0000", 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070703000000003 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261081+0000", 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011867 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261210+0000", 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012960599999999999 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.040 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1407 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.814713531999999, 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044521318999999997, 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1407 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216911+0000", 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.333e-06 2024-09-14T07:08:58.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216916+0000", 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0300000000000001e-06 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216933+0000", 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6616000000000002e-05 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216947+0000", 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3923e-05 2024-09-14T07:08:58.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261260+0000", 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044313105999999998 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261323+0000", 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2688999999999996e-05 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261431+0000", 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108622 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1408 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.814658016999999, 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044764117999999999, 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1408 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:08:58.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216967+0000", 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4360000000000001e-06 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216987+0000", 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9689000000000001e-05 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217005+0000", 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8260000000000001e-05 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217018+0000", 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3396e-05 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261500+0000", 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044481494000000003 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261601+0000", 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101755 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261730+0000", 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128088 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.049 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1561 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.810075130000001, 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044533453000000001, 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1561 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:08:58.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221569+0000", 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-05 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221589+0000", 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9266000000000002e-05 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221602+0000", 2024-09-14T07:08:58.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.366e-05 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265854+0000", 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044251429000000002 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265962+0000", 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266082+0000", 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011999500000000001 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1562 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.809999202999997, 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044744586000000003, 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1562 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221625+0000", 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221631+0000", 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4729999999999999e-06 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221648+0000", 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7346e-05 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221663+0000", 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266141+0000", 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044478651000000001 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266249+0000", 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107371 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266369+0000", 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120258 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1563 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.809941627999997, 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044928442999999998, 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1563 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221683+0000", 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.55e-06 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221689+0000", 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1699999999999996e-06 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221706+0000", 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7286e-05 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221719+0000", 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2959000000000001e-05 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266405+0000", 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044686664000000001 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266488+0000", 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2243999999999994e-05 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266610+0000", 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012256999999999999 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1699 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.062 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.807406796999999, 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044262176, 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1699 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224218+0000", 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224254+0000", 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6525e-05 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224274+0000", 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0049999999999999e-05 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224289+0000", 2024-09-14T07:08:58.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268246+0000", 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043957152999999999 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268354+0000", 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107433 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:08:58.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012504200000000001 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1701 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.807310846999997, 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044338850999999999, 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1701 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224314+0000", 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224320+0000", 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224339+0000", 2024-09-14T07:08:58.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8592000000000001e-05 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224353+0000", 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.432e-05 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268522+0000", 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044169491999999998 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268578+0000", 2024-09-14T07:08:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5099999999999998e-05 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268651+0000", 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-05 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1703 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.807247165, 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044549686999999998, 2024-09-14T07:08:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1703 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224378+0000", 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.573e-06 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224384+0000", 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224403+0000", 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9113000000000001e-05 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224417+0000", 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4049000000000001e-05 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268700+0000", 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044282555000000001 2024-09-14T07:08:58.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268805+0000", 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104858 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268926+0000", 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121296 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1704 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9764864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.807185824000001, 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044649934000000002, 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224439+0000", 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:08:58.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224902+0000", 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046329699999999999 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224924+0000", 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2059e-05 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224941+0000", 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835999999999999e-05 2024-09-14T07:08:58.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268962+0000", 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044020653 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269012+0000", 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0312000000000002e-05 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269088+0000", 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5457000000000006e-05 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1705 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.806653390000001, 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044393046999999998, 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.080 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224972+0000", 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.784e-06 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224979+0000", 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224996+0000", 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225010+0000", 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3926e-05 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269132+0000", 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044121736000000002 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269242+0000", 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010979700000000001 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269363+0000", 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121155 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1706 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.806594228000002, 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044499060999999999, 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.085 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225030+0000", 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225036+0000", 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4530000000000001e-06 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225053+0000", 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6983e-05 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225066+0000", 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269399+0000", 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044332900000000001 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269449+0000", 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9960000000000003e-05 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269528+0000", 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9221999999999995e-05 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1707 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.806539657000002, 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044705944999999997, 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225085+0000", 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225090+0000", 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.113e-06 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225107+0000", 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6569999999999999e-05 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225135+0000", 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8524999999999999e-05 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269571+0000", 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044435781000000001 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269676+0000", 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105374 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269790+0000", 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113415 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1708 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.093 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.806468396, 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044874641, 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:58.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225156+0000", 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225162+0000", 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5960000000000003e-06 2024-09-14T07:08:58.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225179+0000", 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7235999999999999e-05 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225192+0000", 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269839+0000", 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044646439000000003 2024-09-14T07:08:58.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269926+0000", 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7303000000000001e-05 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270030+0000", 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001036 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.097 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1709 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.806412278000003, 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045094769, 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225213+0000", 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6339999999999999e-06 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225218+0000", 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3959999999999997e-06 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225236+0000", 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8053000000000001e-05 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225250+0000", 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3583e-05 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270081+0000", 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044830868000000003 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270179+0000", 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8645000000000002e-05 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270306+0000", 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012658999999999999 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.101 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1710 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.806354702999997, 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045251099000000003, 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:08:58.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225270+0000", 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.173e-06 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225276+0000", 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6840000000000001e-06 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225292+0000", 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6722e-05 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225306+0000", 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3456e-05 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270348+0000", 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045042722 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270431+0000", 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2182999999999996e-05 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270520+0000", 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9159000000000005e-05 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.73475423, 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:58.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:08:58.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:08:58.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.111 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.875278643000001, 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:58.112 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:08:58.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.875176358000001, 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:08:58.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:08:58.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:58.121 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.176124517000002, 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:08:58.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:08:58.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:08:58.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.126 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.126 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.126 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:58.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:08:58.129 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:58.125+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:58.129 INFO:teuthology.orchestra.run.smithi145.stdout:2246 punch from 0x477ee00 to 0x478e800, (0xfa00 bytes) 2024-09-14T07:08:58.129 INFO:teuthology.orchestra.run.smithi145.stdout:2247 read 0x286ee00 thru 0x28721ca (0x33cb bytes) 2024-09-14T07:08:58.129 INFO:teuthology.orchestra.run.smithi145.stdout:2248 trunc from 0x4a14000 to 0xd77aa00 2024-09-14T07:08:58.129 INFO:teuthology.orchestra.run.smithi145.stdout:2249 punch from 0x82a7200 to 0x82b5600, (0xe400 bytes) 2024-09-14T07:08:58.129 INFO:teuthology.orchestra.run.smithi145.stdout:2250 punch from 0xd67a600 to 0xd680a00, (0x6400 bytes) 2024-09-14T07:08:58.129 INFO:teuthology.orchestra.run.smithi145.stdout:2251 read 0x32bee00 thru 0x32c9add (0xacde bytes) 2024-09-14T07:08:58.129 INFO:teuthology.orchestra.run.smithi145.stdout:2252 trunc from 0xd77aa00 to 0x6564a00 2024-09-14T07:08:58.147 INFO:teuthology.orchestra.run.smithi145.stdout:2253 write 0x61fec00 thru 0x62023ca (0x37cb bytes) 2024-09-14T07:08:58.150 INFO:teuthology.orchestra.run.smithi145.stdout:2255 write 0x631f000 thru 0x63249a4 (0x59a5 bytes) 2024-09-14T07:08:58.150 INFO:teuthology.orchestra.run.smithi145.stdout:2256 punch from 0x46abc00 to 0x46ba600, (0xea00 bytes) 2024-09-14T07:08:58.151 INFO:teuthology.orchestra.run.smithi145.stdout:2258 read 0x4f49600 thru 0x4f4a385 (0xd86 bytes) 2024-09-14T07:08:58.179 INFO:teuthology.orchestra.run.smithi145.stdout:2259 trunc from 0x6564a00 to 0xaf3200 2024-09-14T07:08:58.210 INFO:teuthology.orchestra.run.smithi145.stdout:2261 read 0x9e6000 thru 0x9f0e7d (0xae7e bytes) 2024-09-14T07:08:58.213 INFO:teuthology.orchestra.run.smithi145.stdout:2262 trunc from 0xaf3200 to 0xbb0600 2024-09-14T07:08:58.226 INFO:teuthology.orchestra.run.smithi145.stdout:2263 punch from 0x8fa800 to 0x8fbc00, (0x1400 bytes) 2024-09-14T07:08:58.228 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:58.225+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:58.229 INFO:teuthology.orchestra.run.smithi145.stdout:2266 read 0x2b2200 thru 0x2b229c (0x9d bytes) 2024-09-14T07:08:58.248 INFO:teuthology.orchestra.run.smithi145.stdout:2267 write 0x2654000 thru 0x2658fcd (0x4fce bytes) 2024-09-14T07:08:58.248 INFO:teuthology.orchestra.run.smithi145.stdout:2270 punch from 0x1e5fe00 to 0x1e66800, (0x6a00 bytes) 2024-09-14T07:08:58.248 INFO:teuthology.orchestra.run.smithi145.stdout:2271 read 0x6a6200 thru 0x6b2e5d (0xcc5e bytes) 2024-09-14T07:08:58.251 INFO:teuthology.orchestra.run.smithi145.stdout:2272 write 0x1df9800 thru 0x1e0537c (0xbb7d bytes) 2024-09-14T07:08:58.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:08:58.290 INFO:teuthology.orchestra.run.smithi145.stdout:2276 write 0x92bee00 thru 0x92cc006 (0xd207 bytes) 2024-09-14T07:08:58.294 INFO:teuthology.orchestra.run.smithi145.stdout:2280 trunc from 0x92cc200 to 0x862aa00 2024-09-14T07:08:58.315 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:58.316 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:58.316 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:08:58.316 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:58.323 INFO:teuthology.orchestra.run.smithi145.stdout:2281 read 0x5376800 thru 0x537c916 (0x6117 bytes) 2024-09-14T07:08:58.329 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:58.325+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:58.331 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:08:58.339 INFO:teuthology.orchestra.run.smithi145.stdout:2282 trunc from 0x862aa00 to 0xd6cae00 2024-09-14T07:08:58.351 INFO:teuthology.orchestra.run.smithi145.stdout:2285 punch from 0x3e08000 to 0x3e0bc00, (0x3c00 bytes) 2024-09-14T07:08:58.353 INFO:teuthology.orchestra.run.smithi145.stdout:2286 write 0x5100000 thru 0x51095ae (0x95af bytes) 2024-09-14T07:08:58.353 INFO:teuthology.orchestra.run.smithi145.stdout:2287 write 0x3f6da00 thru 0x3f797bb (0xbdbc bytes) 2024-09-14T07:08:58.374 INFO:teuthology.orchestra.run.smithi145.stdout:2288 trunc from 0xd6cae00 to 0x2f09c00 2024-09-14T07:08:58.376 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:08:58.376 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T07:08:58.408 INFO:teuthology.orchestra.run.smithi145.stdout:2289 punch from 0x1e95000 to 0x1e9d000, (0x8000 bytes) 2024-09-14T07:08:58.411 INFO:teuthology.orchestra.run.smithi145.stdout:2290 trunc from 0x2f09c00 to 0x33bb200 2024-09-14T07:08:58.430 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:08:58.425+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:08:58.444 INFO:teuthology.orchestra.run.smithi145.stdout:2292 write 0x4cd0e00 thru 0x4cd2156 (0x1357 bytes) 2024-09-14T07:08:58.471 INFO:teuthology.orchestra.run.smithi145.stdout:2294 write 0x8a22000 thru 0x8a3180e (0xf80f bytes) 2024-09-14T07:08:58.473 INFO:teuthology.orchestra.run.smithi145.stdout:2295 read 0x498a200 thru 0x499890c (0xe70d bytes) 2024-09-14T07:08:58.473 INFO:teuthology.orchestra.run.smithi145.stdout:2296 punch from 0x43d6000 to 0x43dea00, (0x8a00 bytes) 2024-09-14T07:08:58.499 INFO:teuthology.orchestra.run.smithi145.stdout:2297 write 0xcde8800 thru 0xcded87c (0x507d bytes) 2024-09-14T07:08:58.500 INFO:teuthology.orchestra.run.smithi145.stdout:2299 read 0x642ca00 thru 0x643010a (0x370b bytes) 2024-09-14T07:08:58.500 INFO:teuthology.orchestra.run.smithi145.stdout:2300 write 0x787e000 thru 0x7888a82 (0xaa83 bytes) 2024-09-14T07:08:58.500 INFO:teuthology.orchestra.run.smithi145.stdout:2302 write 0x64e2400 thru 0x64e8ec6 (0x6ac7 bytes) 2024-09-14T07:08:58.501 INFO:teuthology.orchestra.run.smithi145.stdout:2303 read 0x8aeee00 thru 0x8afc7a4 (0xd9a5 bytes) 2024-09-14T07:08:58.501 INFO:teuthology.orchestra.run.smithi145.stdout:2304 punch from 0x595d200 to 0x5966c00, (0x9a00 bytes) 2024-09-14T07:08:58.511 INFO:teuthology.orchestra.run.smithi145.stdout:2305 trunc from 0xcdeda00 to 0x6ecfa00 2024-09-14T07:08:58.517 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:08:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:08:58.517 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:08:58.518 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:58.531 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:58.529+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:58.536 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:08:58.561 INFO:teuthology.orchestra.run.smithi145.stdout:2306 trunc from 0x6ecfa00 to 0xdf6ae00 2024-09-14T07:08:58.577 INFO:teuthology.orchestra.run.smithi145.stdout:2310 write 0x506400 thru 0x50b979 (0x557a bytes) 2024-09-14T07:08:58.592 INFO:teuthology.orchestra.run.smithi145.stdout:2311 trunc from 0xdf6ae00 to 0x995dc00 2024-09-14T07:08:58.622 INFO:teuthology.orchestra.run.smithi145.stdout:2312 write 0xa4bec00 thru 0xa4cafbb (0xc3bc bytes) 2024-09-14T07:08:58.624 INFO:teuthology.orchestra.run.smithi145.stdout:2314 write 0x573cc00 thru 0x5748b5a (0xbf5b bytes) 2024-09-14T07:08:58.624 INFO:teuthology.orchestra.run.smithi145.stdout:2315 write 0x6557e00 thru 0x655f224 (0x7425 bytes) 2024-09-14T07:08:58.625 INFO:teuthology.orchestra.run.smithi145.stdout:2317 write 0x8903c00 thru 0x8906901 (0x2d02 bytes) 2024-09-14T07:08:58.632 INFO:teuthology.orchestra.run.smithi145.stdout:2318 trunc from 0xa4cb000 to 0x708d200 2024-09-14T07:08:58.632 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:58.629+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:58.684 INFO:teuthology.orchestra.run.smithi145.stdout:2319 write 0xb6ef000 thru 0xb6fc66d (0xd66e bytes) 2024-09-14T07:08:58.685 INFO:teuthology.orchestra.run.smithi145.stdout:2321 read 0xb643e00 thru 0xb650b39 (0xcd3a bytes) 2024-09-14T07:08:58.719 INFO:teuthology.orchestra.run.smithi145.stdout:2322 write 0xe86ae00 thru 0xe86b7fe (0x9ff bytes) 2024-09-14T07:08:58.720 INFO:teuthology.orchestra.run.smithi145.stdout:2324 read 0xcc97c00 thru 0xcca55fe (0xd9ff bytes) 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:58.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.890139559999994, 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:08:58.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:08:58.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.889365212000001, 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:08:58.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:08:58.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.732 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3038 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.591143037999998, 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046439165999999997, 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.733 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3038 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129535+0000", 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3230000000000001e-06 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129541+0000", 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.054e-06 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129559+0000", 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8329e-05 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129573+0000", 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175474+0000", 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045901019000000001 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175554+0000", 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0661000000000006e-05 2024-09-14T07:08:58.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175973+0000", 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041842300000000002 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3039 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.591085358000001, 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046967035999999997, 2024-09-14T07:08:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3039 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129592+0000", 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.206e-06 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129598+0000", 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2340000000000002e-06 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129616+0000", 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8003e-05 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129629+0000", 2024-09-14T07:08:58.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2913e-05 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176018+0000", 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046389224999999999 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176108+0000", 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176558+0000", 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044997100000000002 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3040 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590996156000003, 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047451337000000003, 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.742 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3040 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129682+0000", 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.607e-06 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129688+0000", 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129708+0000", 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0047e-05 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129723+0000", 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.502e-05 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176597+0000", 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046873259 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176684+0000", 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7349999999999995e-05 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177132+0000", 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044784099999999998 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3041 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590932852999998, 2024-09-14T07:08:58.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047622767000000003, 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3041 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129745+0000", 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129751+0000", 2024-09-14T07:08:58.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129773+0000", 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782999999999999e-05 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129788+0000", 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4606999999999999e-05 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177167+0000", 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047378941000000001 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177253+0000", 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5704999999999993e-05 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177367+0000", 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113861 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.750 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3042 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590869413999997, 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048188976000000001, 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.751 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3042 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129808+0000", 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129814+0000", 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7699999999999998e-06 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129832+0000", 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.751e-05 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129844+0000", 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2797e-05 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177436+0000", 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047591574999999997 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177545+0000", 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109446 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177996+0000", 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045071800000000002 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3043 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590812960000001, 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048376965000000001, 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.755 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3043 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129865+0000", 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129870+0000", 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129888+0000", 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.755e-05 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129902+0000", 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178049+0000", 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048146648 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178131+0000", 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2619000000000003e-05 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178241+0000", 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109262 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3044 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590740881000002, 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048575449, 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3044 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129937+0000", 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1930000000000001e-06 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129943+0000", 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129961+0000", 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8034e-05 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129975+0000", 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3696000000000001e-05 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178293+0000", 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048318051000000001 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178392+0000", 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9038999999999998e-05 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178511+0000", 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119493 2024-09-14T07:08:58.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3045 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590681625000002, 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049079214000000003, 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3045 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129996+0000", 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1170000000000001e-06 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130002+0000", 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130024+0000", 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1763000000000002e-05 2024-09-14T07:08:58.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130059+0000", 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.536e-05 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178555+0000", 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048495599 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178636+0000", 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1112000000000005e-05 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179074+0000", 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043820000000000003 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.768 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3046 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590594949, 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049550174000000002, 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3046 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130083+0000", 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130089+0000", 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130108+0000", 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9046999999999999e-05 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130122+0000", 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4137e-05 2024-09-14T07:08:58.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179126+0000", 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049003927000000003 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179221+0000", 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4717999999999997e-05 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179632+0000", 2024-09-14T07:08:58.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041092199999999998 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3047 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590534251999998, 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050053944000000003, 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.773 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3047 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130144+0000", 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3030000000000001e-06 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130149+0000", 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6230000000000002e-06 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130169+0000", 2024-09-14T07:08:58.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9386999999999999e-05 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130182+0000", 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3706000000000001e-05 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179677+0000", 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049494109000000001 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179763+0000", 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6473999999999999e-05 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180196+0000", 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043334200000000003 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3048 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.777 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590474333000003, 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050582655999999997, 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3048 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130204+0000", 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.266e-06 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130219+0000", 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5549999999999999e-05 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130235+0000", 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6030000000000001e-05 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130249+0000", 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3977e-05 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180264+0000", 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050014851999999999 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180359+0000", 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5400000000000001e-05 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180785+0000", 2024-09-14T07:08:58.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042558100000000002 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3049 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.590406866999999, 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.051156504999999998, 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3049 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130271+0000", 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130277+0000", 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6370000000000004e-06 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130297+0000", 2024-09-14T07:08:58.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.997e-05 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130311+0000", 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4358999999999999e-05 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180833+0000", 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050521863 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180936+0000", 2024-09-14T07:08:58.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010296799999999999 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.181426+0000", 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049044799999999999 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3149 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.786 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.575220291000001, 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046930569999999998, 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3149 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:58.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145458+0000", 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145463+0000", 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6169999999999998e-06 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145486+0000", 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2626e-05 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145500+0000", 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4243000000000001e-05 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191669+0000", 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046169090000000003 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191811+0000", 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014187100000000001 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192387+0000", 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057567299999999996 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3150 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.575153931000003, 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047680769999999997, 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3150 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.791 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145524+0000", 2024-09-14T07:08:58.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.39e-06 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145647+0000", 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123242 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145673+0000", 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145711+0000", 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8745999999999998e-05 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192444+0000", 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046732462000000002 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192602+0000", 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015825400000000001 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193203+0000", 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00060150600000000002 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3172 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.574768061, 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048262865000000002, 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.795 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3172 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:08:58.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145911+0000", 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145920+0000", 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2969999999999992e-06 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145947+0000", 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7149999999999999e-05 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145965+0000", 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8295999999999999e-05 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193303+0000", 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047338247 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193492+0000", 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018843500000000001 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194171+0000", 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067951700000000003 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3173 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:58.799 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.574682705000001, 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048937952999999999, 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3173 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:58.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145995+0000", 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146021+0000", 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5684e-05 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146044+0000", 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2503000000000001e-05 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146059+0000", 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5043e-05 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194258+0000", 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04819975 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194372+0000", 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113373 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194932+0000", 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056004399999999999 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.878182213999999, 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:08:58.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.809 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 36.581254008000002, 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:08:58.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:08:58.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:08:58.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:08:58.815 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:08:58.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:08:58.818 INFO:teuthology.orchestra.run.smithi145.stdout:2325 trunc from 0xe86b800 to 0xc83e800 2024-09-14T07:08:58.818 INFO:teuthology.orchestra.run.smithi145.stdout:2327 trunc from 0xc83e800 to 0xcede400 2024-09-14T07:08:58.818 INFO:teuthology.orchestra.run.smithi145.stdout:2328 trunc from 0xcede400 to 0xd8fbe00 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2329 read 0xd817e00 thru 0xd81a785 (0x2986 bytes) 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2331 trunc from 0xd8fbe00 to 0x89d0000 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2333 punch from 0x602000 to 0x609200, (0x7200 bytes) 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2334 write 0x82fd600 thru 0x83015cb (0x3fcc bytes) 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2338 write 0x7168000 thru 0x7170ee6 (0x8ee7 bytes) 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2342 punch from 0x1c3ea00 to 0x1c41800, (0x2e00 bytes) 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2344 punch from 0x1f85c00 to 0x1f88600, (0x2a00 bytes) 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2345 read 0x312c800 thru 0x312ef72 (0x2773 bytes) 2024-09-14T07:08:58.819 INFO:teuthology.orchestra.run.smithi145.stdout:2346 write 0x514400 thru 0x51adee (0x69ef bytes) 2024-09-14T07:08:58.820 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:58.813+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:58.820 INFO:teuthology.orchestra.run.smithi145.stdout:2347 read 0x491400 thru 0x4a059c (0xf19d bytes) 2024-09-14T07:08:58.822 INFO:teuthology.orchestra.run.smithi145.stdout:2348 read 0x8875400 thru 0x8878c19 (0x381a bytes) 2024-09-14T07:08:58.822 INFO:teuthology.orchestra.run.smithi145.stdout:2350 read 0x6213200 thru 0x622038c (0xd18d bytes) 2024-09-14T07:08:58.825 INFO:teuthology.orchestra.run.smithi145.stdout:2351 trunc from 0x89d0000 to 0x7776200 2024-09-14T07:08:58.864 INFO:teuthology.orchestra.run.smithi145.stdout:2352 trunc from 0x7776200 to 0xb6f000 2024-09-14T07:08:58.905 INFO:teuthology.orchestra.run.smithi145.stdout:2353 read 0x2e9a00 thru 0x2f8150 (0xe751 bytes) 2024-09-14T07:08:58.908 INFO:teuthology.orchestra.run.smithi145.stdout:2355 read 0x70a800 thru 0x70fda8 (0x55a9 bytes) 2024-09-14T07:08:58.909 INFO:teuthology.orchestra.run.smithi145.stdout:2357 punch from 0x647200 to 0x647600, (0x400 bytes) 2024-09-14T07:08:58.910 INFO:teuthology.orchestra.run.smithi145.stdout:2360 read 0x898800 thru 0x8a0dd7 (0x85d8 bytes) 2024-09-14T07:08:58.918 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:58.913+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:58.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:08:58.937 INFO:teuthology.orchestra.run.smithi145.stdout:2364 trunc from 0xb6f000 to 0xa7d3600 2024-09-14T07:08:58.956 INFO:teuthology.orchestra.run.smithi145.stdout:2365 read 0x5c8c200 thru 0x5c91392 (0x5193 bytes) 2024-09-14T07:08:58.961 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:08:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:08:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:08:58.961 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:08:58.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:08:58.983 INFO:teuthology.orchestra.run.smithi145.stdout:2367 write 0xe49a200 thru 0xe4a91df (0xefe0 bytes) 2024-09-14T07:08:58.994 INFO:teuthology.orchestra.run.smithi145.stdout:2370 trunc from 0xe4a9200 to 0xa400c00 2024-09-14T07:08:59.019 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:08:59.012+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:08:59.034 INFO:teuthology.orchestra.run.smithi145.stdout:2377 punch from 0x6aa7600 to 0x6aa9e00, (0x2800 bytes) 2024-09-14T07:08:59.037 INFO:teuthology.orchestra.run.smithi145.stdout:2378 write 0x1b36a00 thru 0x1b3996e (0x2f6f bytes) 2024-09-14T07:08:59.037 INFO:teuthology.orchestra.run.smithi145.stdout:2379 punch from 0x3c5ce00 to 0x3c5e000, (0x1200 bytes) 2024-09-14T07:08:59.038 INFO:teuthology.orchestra.run.smithi145.stdout:2380 write 0x4752c00 thru 0x4755847 (0x2c48 bytes) 2024-09-14T07:08:59.038 INFO:teuthology.orchestra.run.smithi145.stdout:2382 write 0x5897400 thru 0x58a14a9 (0xa0aa bytes) 2024-09-14T07:08:59.038 INFO:teuthology.orchestra.run.smithi145.stdout:2383 read 0x2107e00 thru 0x2112275 (0xa476 bytes) 2024-09-14T07:08:59.038 INFO:teuthology.orchestra.run.smithi145.stdout:2384 read 0x76d1e00 thru 0x76d5452 (0x3653 bytes) 2024-09-14T07:08:59.068 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:08:59.068 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:08:59.073 INFO:teuthology.orchestra.run.smithi145.stdout:2385 write 0xad90200 thru 0xad94127 (0x3f28 bytes) 2024-09-14T07:08:59.074 INFO:teuthology.orchestra.run.smithi145.stdout:2389 punch from 0x4619800 to 0x4622e00, (0x9600 bytes) 2024-09-14T07:08:59.075 INFO:teuthology.orchestra.run.smithi145.stdout:2390 write 0x70af600 thru 0x70b989a (0xa29b bytes) 2024-09-14T07:08:59.075 INFO:teuthology.orchestra.run.smithi145.stdout:2392 write 0x90c3400 thru 0x90cf9b8 (0xc5b9 bytes) 2024-09-14T07:08:59.076 INFO:teuthology.orchestra.run.smithi145.stdout:2394 write 0x28a9400 thru 0x28b2d9d (0x999e bytes) 2024-09-14T07:08:59.076 INFO:teuthology.orchestra.run.smithi145.stdout:2395 punch from 0x3b47800 to 0x3b55400, (0xdc00 bytes) 2024-09-14T07:08:59.076 INFO:teuthology.orchestra.run.smithi145.stdout:2398 write 0x5ccaa00 thru 0x5ccde54 (0x3455 bytes) 2024-09-14T07:08:59.076 INFO:teuthology.orchestra.run.smithi145.stdout:2399 read 0x40e4c00 thru 0x40f0eea (0xc2eb bytes) 2024-09-14T07:08:59.082 INFO:teuthology.orchestra.run.smithi145.stdout:2401 trunc from 0xad94200 to 0x8998400 2024-09-14T07:08:59.087 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:08:59.087 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:08:59.105 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:08:59.106 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:08:59.120 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:59.117+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:59.124 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:08:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:08:59.139 INFO:teuthology.orchestra.run.smithi145.stdout:2405 write 0xaaaf600 thru 0xaab52c6 (0x5cc7 bytes) 2024-09-14T07:08:59.141 INFO:teuthology.orchestra.run.smithi145.stdout:2406 write 0x6881c00 thru 0x68886b6 (0x6ab7 bytes) 2024-09-14T07:08:59.141 INFO:teuthology.orchestra.run.smithi145.stdout:2407 read 0x3dc7000 thru 0x3dcd4ea (0x64eb bytes) 2024-09-14T07:08:59.142 INFO:teuthology.orchestra.run.smithi145.stdout:2411 read 0x21a5e00 thru 0x21b356f (0xd770 bytes) 2024-09-14T07:08:59.142 INFO:teuthology.orchestra.run.smithi145.stdout:2412 write 0x6947400 thru 0x694d299 (0x5e9a bytes) 2024-09-14T07:08:59.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:08:59.144 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:08:59.145 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:59.145 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:59.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:08:59.160 INFO:teuthology.orchestra.run.smithi145.stdout:2416 trunc from 0xaab5400 to 0x401d400 2024-09-14T07:08:59.166 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:08:59.166 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:08:59.202 INFO:teuthology.orchestra.run.smithi145.stdout:2418 punch from 0x30c6000 to 0x30ce600, (0x8600 bytes) 2024-09-14T07:08:59.204 INFO:teuthology.orchestra.run.smithi145.stdout:2419 read 0x1428800 thru 0x142cb63 (0x4364 bytes) 2024-09-14T07:08:59.204 INFO:teuthology.orchestra.run.smithi145.stdout:2420 read 0x31c6000 thru 0x31cd211 (0x7212 bytes) 2024-09-14T07:08:59.205 INFO:teuthology.orchestra.run.smithi145.stdout:2421 clone 16 order 25 su 2097152 sc 6 2024-09-14T07:08:59.231 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:59.229+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:59.263 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:08:59.265 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T07:08:59.290 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:59.290 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:59.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T07:08:59.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:08:59.348 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:08:59.345+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:08:59.419 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone15 from 0x401d400 (overlap 0xaf3200) to 0x348e00 2024-09-14T07:08:59.423 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:59.421+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:59.475 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:59.475 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:59.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:08:59.535 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:59.533+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:59.624 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:59.621+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:59.653 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:59.653 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:59.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:08:59.759 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:08:59.757+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:08:59.825 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:08:59.821+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:08:59.838 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:08:59.838 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:08:59.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:08:59.926 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:08:59.920+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:08:59.949 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T07:09:00.026 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:00.021+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:00.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:00.126 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:00.120+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:00.227 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:00.224+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:00.238 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:00.238 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:00.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:00.264 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T07:09:00.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T07:09:00.339 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:00.337+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:00.398 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:00.398 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:00.416 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:00.428 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:00.425+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:00.530 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:00.524+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:00.578 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:00.578 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:00.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:00.630 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:00.625+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:00.731 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:00.729+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:00.782 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:00.783 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:00.799 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:00.832 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:00.829+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:00.918 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:00.918 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:00.932 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:00.933 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:00.929+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:01.034 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:01.028+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:01.057 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:01.057 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:01.058 INFO:teuthology.orchestra.run.smithi145.stdout:2422 read 0x15ef600 thru 0x15f219a (0x2b9b bytes) 2024-09-14T07:09:01.071 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:01.095 INFO:teuthology.orchestra.run.smithi145.stdout:2423 trunc from 0x401d400 to 0xabc8600 2024-09-14T07:09:01.125 INFO:teuthology.orchestra.run.smithi145.stdout:2424 flatten 2024-09-14T07:09:01.161 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:01.156+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:01.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:01.236 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:01.232+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:01.242 INFO:teuthology.orchestra.run.smithi145.stdout:2427 write 0x7ee3000 thru 0x7eef466 (0xc467 bytes) 2024-09-14T07:09:01.243 INFO:teuthology.orchestra.run.smithi145.stdout:2428 clone 17 order 24 su 1048576 sc 12 2024-09-14T07:09:01.266 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:01.266 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:01.274 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T07:09:01.282 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:01.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:09:01.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T07:09:01.362 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:01.362 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:01.371 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:01.368+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:01.380 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:01.380 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:01.397 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:01.397 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:01.418 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:01.437 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:01.432+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:01.439 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:01.457 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:01.457 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1404 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12124160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.238196541000001, 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044358094000000001, 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1404 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216852+0000", 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216853+0000", 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699999999999999e-06 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216859+0000", 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2669999999999997e-06 2024-09-14T07:09:01.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216877+0000", 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7866000000000001e-05 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216891+0000", 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4712000000000001e-05 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.260962+0000", 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044070703000000003 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261081+0000", 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011867 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261210+0000", 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012960599999999999 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1407 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 12713984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.238138796000001, 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044521318999999997, 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1407 2024-09-14T07:09:01.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216910+0000", 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216911+0000", 2024-09-14T07:09:01.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.333e-06 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216916+0000", 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0300000000000001e-06 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216933+0000", 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6616000000000002e-05 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216947+0000", 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3923e-05 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261260+0000", 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044313105999999998 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261323+0000", 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2688999999999996e-05 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261431+0000", 2024-09-14T07:09:01.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108622 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1408 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13303808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.238083281000002, 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044764117999999999, 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1408 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216965+0000", 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216967+0000", 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4360000000000001e-06 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.216987+0000", 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9689000000000001e-05 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217005+0000", 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8260000000000001e-05 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.217018+0000", 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3396e-05 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261500+0000", 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044481494000000003 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261601+0000", 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101755 2024-09-14T07:09:01.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.261730+0000", 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128088 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1561 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.233500394000004, 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044533453000000001, 2024-09-14T07:09:01.471 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1561 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221548+0000", 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221549+0000", 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221569+0000", 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-05 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221589+0000", 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9266000000000002e-05 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221602+0000", 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.366e-05 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265854+0000", 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044251429000000002 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.265962+0000", 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010814700000000001 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266082+0000", 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011999500000000001 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1562 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.475 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.233424466999999, 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044744586000000003, 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1562 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:09:01.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221624+0000", 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221625+0000", 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221631+0000", 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4729999999999999e-06 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221648+0000", 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7346e-05 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221663+0000", 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266141+0000", 2024-09-14T07:09:01.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044478651000000001 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266249+0000", 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107371 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266369+0000", 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120258 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1563 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 15400960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.233366891999999, 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044928442999999998, 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.480 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1563 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221682+0000", 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221683+0000", 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.55e-06 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221689+0000", 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1699999999999996e-06 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221706+0000", 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7286e-05 2024-09-14T07:09:01.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.221719+0000", 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2959000000000001e-05 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266405+0000", 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044686664000000001 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266488+0000", 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2243999999999994e-05 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.266610+0000", 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012256999999999999 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1699 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 6422528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.230832061000001, 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044262176, 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.484 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1699 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224217+0000", 2024-09-14T07:09:01.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224218+0000", 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224254+0000", 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6525e-05 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224274+0000", 2024-09-14T07:09:01.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0049999999999999e-05 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224289+0000", 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268246+0000", 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043957152999999999 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268354+0000", 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107433 2024-09-14T07:09:01.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268479+0000", 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012504200000000001 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1701 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.230736110999999, 2024-09-14T07:09:01.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044338850999999999, 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1701 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224313+0000", 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224314+0000", 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224320+0000", 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1569999999999997e-06 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224339+0000", 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8592000000000001e-05 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224353+0000", 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.432e-05 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268522+0000", 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044169491999999998 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268578+0000", 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5099999999999998e-05 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268651+0000", 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3889999999999999e-05 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1703 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9175040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.230672429000002, 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044549686999999998, 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1703 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224376+0000", 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224378+0000", 2024-09-14T07:09:01.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.573e-06 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224384+0000", 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224403+0000", 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9113000000000001e-05 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224417+0000", 2024-09-14T07:09:01.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4049000000000001e-05 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268700+0000", 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044282555000000001 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268805+0000", 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104858 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268926+0000", 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121296 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1704 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 9764864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.230611088000003, 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044649934000000002, 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224438+0000", 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224439+0000", 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224902+0000", 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046329699999999999 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224924+0000", 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2059e-05 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224941+0000", 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835999999999999e-05 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.268962+0000", 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044020653 2024-09-14T07:09:01.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269012+0000", 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0312000000000002e-05 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269088+0000", 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5457000000000006e-05 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1705 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10354688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.230078654000003, 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044393046999999998, 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224970+0000", 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224972+0000", 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.784e-06 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224979+0000", 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.224996+0000", 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225010+0000", 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3926e-05 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269132+0000", 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044121736000000002 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269242+0000", 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010979700000000001 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269363+0000", 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121155 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.505 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1706 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 10944512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.230019491999997, 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044499060999999999, 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.506 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225029+0000", 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225030+0000", 2024-09-14T07:09:01.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225036+0000", 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4530000000000001e-06 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225053+0000", 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6983e-05 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225066+0000", 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:09:01.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269399+0000", 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044332900000000001 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269449+0000", 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9960000000000003e-05 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269528+0000", 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9221999999999995e-05 2024-09-14T07:09:01.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1707 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13107200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.229964920999997, 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044705944999999997, 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:01.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225084+0000", 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225085+0000", 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:09:01.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225090+0000", 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.113e-06 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225107+0000", 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6569999999999999e-05 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225135+0000", 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8524999999999999e-05 2024-09-14T07:09:01.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269571+0000", 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044435781000000001 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269676+0000", 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105374 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269790+0000", 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113415 2024-09-14T07:09:01.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1708 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 13697024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.229893660000002, 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044874641, 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:01.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225155+0000", 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225156+0000", 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:09:01.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225162+0000", 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5960000000000003e-06 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225179+0000", 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7235999999999999e-05 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225192+0000", 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.327e-05 2024-09-14T07:09:01.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269839+0000", 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044646439000000003 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.269926+0000", 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7303000000000001e-05 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270030+0000", 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001036 2024-09-14T07:09:01.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1709 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14286848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.229837541999999, 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045094769, 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:01.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225211+0000", 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225213+0000", 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6339999999999999e-06 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225218+0000", 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3959999999999997e-06 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225236+0000", 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8053000000000001e-05 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225250+0000", 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3583e-05 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270081+0000", 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044830868000000003 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270179+0000", 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8645000000000002e-05 2024-09-14T07:09:01.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270306+0000", 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012658999999999999 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1710 3.1a 3:5d5d9945:::rbd_data.116167e24a2d.0000000000000007:a [sparse-read 14876672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.229779966999999, 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045251099000000003, 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225269+0000", 2024-09-14T07:09:01.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225270+0000", 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.173e-06 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225276+0000", 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6840000000000001e-06 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225292+0000", 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6722e-05 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.225306+0000", 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3456e-05 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270348+0000", 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045042722 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270431+0000", 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2182999999999996e-05 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.270520+0000", 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9159000000000005e-05 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:01.526 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 44.158179494000002, 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:01.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:01.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:01.532 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.298703907, 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:01.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:01.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:01.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.298601622, 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:01.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:01.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 30.599549781, 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:01.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:01.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:01.546 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:01.547 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:01.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:01.549 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:01.549 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:01.544+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:01.648 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:01.644+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:01.723 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:01.723 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:01.723 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:01.723 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:01.737 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:01.749 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:01.744+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:01.850 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:01.844+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:01.861 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:01.861 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:01.861 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:01.861 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:01.861 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:01.876 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:01.931 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T07:09:01.951 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:01.948+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:02.007 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:02.007 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:02.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:02.008 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:02.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.008 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:02.008 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:02.008 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.175733794999999, 2024-09-14T07:09:02.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:02.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:02.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.174959447000006, 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:02.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:02.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3038 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876737273000003, 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046439165999999997, 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.019 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3038 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129534+0000", 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129535+0000", 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3230000000000001e-06 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129541+0000", 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.054e-06 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129559+0000", 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8329e-05 2024-09-14T07:09:02.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129573+0000", 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175474+0000", 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045901019000000001 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175554+0000", 2024-09-14T07:09:02.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0661000000000006e-05 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.175973+0000", 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041842300000000002 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3039 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 131072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876679592999999, 2024-09-14T07:09:02.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046967035999999997, 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3039 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:09:02.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129591+0000", 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129592+0000", 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.206e-06 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129598+0000", 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2340000000000002e-06 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129616+0000", 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8003e-05 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129629+0000", 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2913e-05 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176018+0000", 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046389224999999999 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176108+0000", 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0483999999999996e-05 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176558+0000", 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044997100000000002 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.027 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3040 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876590391000001, 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047451337000000003, 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.028 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3040 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129680+0000", 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129682+0000", 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.607e-06 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129688+0000", 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129708+0000", 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0047e-05 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129723+0000", 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.502e-05 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176597+0000", 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046873259 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.176684+0000", 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7349999999999995e-05 2024-09-14T07:09:02.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177132+0000", 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00044784099999999998 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3041 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 188416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876527088000003, 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047622767000000003, 2024-09-14T07:09:02.032 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3041 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129744+0000", 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129745+0000", 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129751+0000", 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129773+0000", 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782999999999999e-05 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129788+0000", 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4606999999999999e-05 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177167+0000", 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047378941000000001 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177253+0000", 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5704999999999993e-05 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177367+0000", 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113861 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.036 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3042 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 147456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876463649000002, 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048188976000000001, 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3042 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129807+0000", 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129808+0000", 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129814+0000", 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7699999999999998e-06 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129832+0000", 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.751e-05 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129844+0000", 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2797e-05 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177436+0000", 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047591574999999997 2024-09-14T07:09:02.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177545+0000", 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109446 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.177996+0000", 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045071800000000002 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3043 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 245760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876407194999999, 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048376965000000001, 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.042 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3043 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129864+0000", 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129865+0000", 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129870+0000", 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129888+0000", 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.755e-05 2024-09-14T07:09:02.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129902+0000", 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4399999999999999e-05 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178049+0000", 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048146648 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178131+0000", 2024-09-14T07:09:02.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2619000000000003e-05 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178241+0000", 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109262 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3044 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 204800~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876335116, 2024-09-14T07:09:02.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048575449, 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3044 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129936+0000", 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129937+0000", 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1930000000000001e-06 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129943+0000", 2024-09-14T07:09:02.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9429999999999997e-06 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129961+0000", 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8034e-05 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129975+0000", 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3696000000000001e-05 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178293+0000", 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048318051000000001 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178392+0000", 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9038999999999998e-05 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178511+0000", 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119493 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.050 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3045 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 262144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.87627586, 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049079214000000003, 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.051 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3045 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129995+0000", 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.129996+0000", 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1170000000000001e-06 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130002+0000", 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130024+0000", 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1763000000000002e-05 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130059+0000", 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.536e-05 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178555+0000", 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048495599 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.178636+0000", 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1112000000000005e-05 2024-09-14T07:09:02.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179074+0000", 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043820000000000003 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3046 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 303104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876189183999998, 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049550174000000002, 2024-09-14T07:09:02.055 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3046 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130082+0000", 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130083+0000", 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130089+0000", 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130108+0000", 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9046999999999999e-05 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130122+0000", 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4137e-05 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179126+0000", 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049003927000000003 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179221+0000", 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4717999999999997e-05 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179632+0000", 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041092199999999998 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3047 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 319488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.059 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876128487000003, 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050053944000000003, 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3047 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130142+0000", 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130144+0000", 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3030000000000001e-06 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130149+0000", 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6230000000000002e-06 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130169+0000", 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9386999999999999e-05 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130182+0000", 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3706000000000001e-05 2024-09-14T07:09:02.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179677+0000", 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049494109000000001 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.179763+0000", 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6473999999999999e-05 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180196+0000", 2024-09-14T07:09:02.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043334200000000003 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3048 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 376832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876068568000001, 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050582655999999997, 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3048 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130202+0000", 2024-09-14T07:09:02.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130204+0000", 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.266e-06 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130219+0000", 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5549999999999999e-05 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130235+0000", 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6030000000000001e-05 2024-09-14T07:09:02.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130249+0000", 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3977e-05 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180264+0000", 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050014851999999999 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180359+0000", 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5400000000000001e-05 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180785+0000", 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042558100000000002 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3049 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.876001102000004, 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.051156504999999998, 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.068 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3049 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130270+0000", 2024-09-14T07:09:02.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130271+0000", 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130277+0000", 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6370000000000004e-06 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130297+0000", 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.997e-05 2024-09-14T07:09:02.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.130311+0000", 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4358999999999999e-05 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180833+0000", 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050521863 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.180936+0000", 2024-09-14T07:09:02.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010296799999999999 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.181426+0000", 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049044799999999999 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3149 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.860814525999999, 2024-09-14T07:09:02.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046930569999999998, 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3149 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145456+0000", 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145458+0000", 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145463+0000", 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6169999999999998e-06 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145486+0000", 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2626e-05 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145500+0000", 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4243000000000001e-05 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191669+0000", 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046169090000000003 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.191811+0000", 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014187100000000001 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192387+0000", 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057567299999999996 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3150 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 57344~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:09:02.076 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.860748166, 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047680769999999997, 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3150 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145523+0000", 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145524+0000", 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.39e-06 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145647+0000", 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123242 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145673+0000", 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145711+0000", 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8745999999999998e-05 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192444+0000", 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046732462000000002 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.192602+0000", 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015825400000000001 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193203+0000", 2024-09-14T07:09:02.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00060150600000000002 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3172 3.5 3:a33e595a:::rbd_data.1161dbabc1ae.0000000000000015:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.860362295999998, 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048262865000000002, 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3172 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145909+0000", 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145911+0000", 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145920+0000", 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2969999999999992e-06 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145947+0000", 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7149999999999999e-05 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145965+0000", 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8295999999999999e-05 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193303+0000", 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.047338247 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.193492+0000", 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018843500000000001 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194171+0000", 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067951700000000003 2024-09-14T07:09:02.085 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4579.0:3173 3.5 3:a7f2e27d:::rbd_data.1161dbabc1ae.0000000000000016:8 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.860276939999999, 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.048937952999999999, 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4579", 2024-09-14T07:09:02.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3728866361", 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3173 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145994+0000", 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.145995+0000", 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146021+0000", 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5684e-05 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146044+0000", 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2503000000000001e-05 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.146059+0000", 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5043e-05 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194258+0000", 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04819975 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194372+0000", 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000113373 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:13.194932+0000", 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056004399999999999 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.163776448999997, 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:02.090 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:02.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:02.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 39.866848243, 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:02.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:02.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:02.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:02.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:02.103 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:02.103 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:02.104 INFO:teuthology.orchestra.run.smithi145.stdout:2429 write 0xe077000 thru 0xe082d6c (0xbd6d bytes) 2024-09-14T07:09:02.104 INFO:teuthology.orchestra.run.smithi145.stdout:2431 read 0x6311400 thru 0x631cb09 (0xb70a bytes) 2024-09-14T07:09:02.104 INFO:teuthology.orchestra.run.smithi145.stdout:2432 write 0x44e8600 thru 0x44f589b (0xd29c bytes) 2024-09-14T07:09:02.104 INFO:teuthology.orchestra.run.smithi145.stdout:2435 trunc from 0xe082e00 to 0xc8bca00 2024-09-14T07:09:02.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:02.105 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:02.100+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:02.135 INFO:teuthology.orchestra.run.smithi145.stdout:2436 punch from 0x2801e00 to 0x280d000, (0xb200 bytes) 2024-09-14T07:09:02.161 INFO:teuthology.orchestra.run.smithi145.stdout:2437 trunc from 0xc8bca00 to 0x9896600 2024-09-14T07:09:02.205 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:02.200+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:02.252 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:02.253 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:02.253 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:02.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:02.274 INFO:teuthology.orchestra.run.smithi145.stdout:2438 read 0x6327e00 thru 0x6336112 (0xe313 bytes) 2024-09-14T07:09:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T07:09:02.281 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:09:02.296 INFO:teuthology.orchestra.run.smithi145.stdout:2439 trunc from 0x9896600 to 0xe6da200 2024-09-14T07:09:02.308 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:02.304+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:02.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T07:09:02.315 INFO:teuthology.orchestra.run.smithi145.stdout:2440 read 0x4dc7200 thru 0x4dd01b2 (0x8fb3 bytes) 2024-09-14T07:09:02.317 INFO:teuthology.orchestra.run.smithi145.stdout:2442 write 0x75da800 thru 0x75e02e9 (0x5aea bytes) 2024-09-14T07:09:02.328 INFO:teuthology.orchestra.run.smithi145.stdout:2444 trunc from 0xe6da200 to 0xae1d600 2024-09-14T07:09:02.357 INFO:teuthology.orchestra.run.smithi145.stdout:2445 read 0x1e4ec00 thru 0x1e5b2af (0xc6b0 bytes) 2024-09-14T07:09:02.359 INFO:teuthology.orchestra.run.smithi145.stdout:2447 flatten 2024-09-14T07:09:02.375 INFO:teuthology.orchestra.run.smithi145.stdout:2449 write 0x1e0ce00 thru 0x1e1b59d (0xe79e bytes) 2024-09-14T07:09:02.376 INFO:teuthology.orchestra.run.smithi145.stdout:2450 read 0x2c0b600 thru 0x2c0fb0d (0x450e bytes) 2024-09-14T07:09:02.377 INFO:teuthology.orchestra.run.smithi145.stdout:2451 write 0x76b5800 thru 0x76b94ad (0x3cae bytes) 2024-09-14T07:09:02.406 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:02.400+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:02.411 INFO:teuthology.orchestra.run.smithi145.stdout:2452 write 0xe625400 thru 0xe62ea2a (0x962b bytes) 2024-09-14T07:09:02.413 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:02.413 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:02.413 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:02.413 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:02.414 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:02.414 INFO:teuthology.orchestra.run.smithi145.stdout:2453 read 0x8527800 thru 0x8536130 (0xe931 bytes) 2024-09-14T07:09:02.415 INFO:teuthology.orchestra.run.smithi145.stdout:2454 read 0xf97200 thru 0xf98cfd (0x1afe bytes) 2024-09-14T07:09:02.416 INFO:teuthology.orchestra.run.smithi145.stdout:2455 write 0x500aa00 thru 0x50127a9 (0x7daa bytes) 2024-09-14T07:09:02.418 INFO:teuthology.orchestra.run.smithi145.stdout:2456 write 0x51d4400 thru 0x51e08d7 (0xc4d8 bytes) 2024-09-14T07:09:02.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:02.507 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:02.504+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:02.508 INFO:teuthology.orchestra.run.smithi145.stdout:2460 trunc from 0xe62ec00 to 0x23bae00 2024-09-14T07:09:02.556 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.266259353000002, 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.236751722000001, 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.228104363, 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.562012023000001, 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:02.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:02.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.395282797, 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:02.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:02.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.213351283000002, 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:09:02.580 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:09:02.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.462281789000002, 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:02.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:02.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:02.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:02.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4732.0:183 3.1b 3:d96d4dd6:::rbd_data.1161bf39a192.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 6709248~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.612411749000003, 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030801473999999999, 2024-09-14T07:09:02.590 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4732", 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/303097868", 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 183 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942590+0000", 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944540+0000", 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001949824 2024-09-14T07:09:02.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944564+0000", 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4199e-05 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944574+0000", 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944644+0000", 2024-09-14T07:09:02.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0240999999999994e-05 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944798+0000", 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153432 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.945082+0000", 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028403899999999998 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.959401+0000", 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014319159999999999 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973083+0000", 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013682016 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973261+0000", 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017752999999999999 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973390+0000", 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012949500000000001 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:538 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.62072612, 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020569724000000001, 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 538 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934275+0000", 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934281+0000", 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934299+0000", 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8470000000000001e-05 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934313+0000", 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3196e-05 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954450+0000", 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020137616000000001 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954629+0000", 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017857499999999999 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954844+0000", 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021515099999999999 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:539 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620604369999999, 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02104733, 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 539 2024-09-14T07:09:02.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934398+0000", 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.813e-06 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934404+0000", 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7460000000000003e-06 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934425+0000", 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.033e-05 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934440+0000", 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5019e-05 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954916+0000", 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020476173 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955086+0000", 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169655 2024-09-14T07:09:02.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955443+0000", 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035759399999999998 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:540 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620536775000001, 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021617992999999999, 2024-09-14T07:09:02.605 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 540 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934465+0000", 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4130000000000001e-06 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934471+0000", 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7259999999999999e-06 2024-09-14T07:09:02.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934489+0000", 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8263000000000001e-05 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934506+0000", 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955539+0000", 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021033046999999999 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955813+0000", 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00027412200000000002 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956082+0000", 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000268829 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:541 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620471837, 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022127142999999998, 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 541 2024-09-14T07:09:02.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934530+0000", 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.037e-06 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934535+0000", 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8259999999999998e-06 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934554+0000", 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934567+0000", 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3573e-05 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956183+0000", 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021615059999999998 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956395+0000", 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021209399999999999 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956656+0000", 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026098000000000001 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:542 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:02.614 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620412846000001, 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022604783999999999, 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 542 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:02.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934589+0000", 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1039999999999999e-06 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934594+0000", 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.293e-06 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934612+0000", 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8578999999999999e-05 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934626+0000", 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3613e-05 2024-09-14T07:09:02.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956738+0000", 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022111927 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956951+0000", 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021302100000000001 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957192+0000", 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024124699999999999 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:543 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620355304, 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025291999999999, 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 543 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:02.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934646+0000", 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934652+0000", 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000004e-06 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934670+0000", 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.808e-05 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934683+0000", 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3686000000000001e-05 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957298+0000", 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02261471 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957448+0000", 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015033900000000001 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957670+0000", 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022180100000000001 2024-09-14T07:09:02.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:544 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620297679, 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023570109999999998, 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 544 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934704+0000", 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934726+0000", 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2005999999999999e-05 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934744+0000", 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8297000000000001e-05 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934788+0000", 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3631999999999998e-05 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957813+0000", 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025417999999999 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958049+0000", 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023578 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958273+0000", 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022392700000000001 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:545 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620184498, 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023972594999999999, 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 545 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934817+0000", 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3060000000000001e-06 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934823+0000", 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-06 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934843+0000", 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9576e-05 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934859+0000", 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.588e-05 2024-09-14T07:09:02.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958368+0000", 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023509124999999999 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958487+0000", 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000119123 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958788+0000", 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030121500000000002 2024-09-14T07:09:02.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:546 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620083731000001, 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024419039, 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 546 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934918+0000", 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934925+0000", 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9739999999999996e-06 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934946+0000", 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0528999999999999e-05 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934960+0000", 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4343e-05 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958881+0000", 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023921114 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959100+0000", 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021872699999999999 2024-09-14T07:09:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959336+0000", 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023562600000000001 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:547 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.620017423, 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024711538000000002, 2024-09-14T07:09:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 547 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934984+0000", 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.513e-06 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934990+0000", 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5929999999999996e-06 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935008+0000", 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7802999999999999e-05 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935021+0000", 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3506000000000001e-05 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959426+0000", 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024404250999999998 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959538+0000", 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112534 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959694+0000", 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015633800000000001 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:548 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.619955052000002, 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024957958999999998, 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 548 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935046+0000", 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.147e-06 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935052+0000", 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6130000000000003e-06 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935071+0000", 2024-09-14T07:09:02.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8522999999999999e-05 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935084+0000", 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3823e-05 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959754+0000", 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024669673 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959864+0000", 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109757 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.960003+0000", 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013942300000000001 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:591 3.1f 3:f9b02d91:::rbd_data.1161605482a1.0000000000000004:e [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.619000488000001, 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020703709000000001, 2024-09-14T07:09:02.646 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 591 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936001+0000", 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.083e-06 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936006+0000", 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-09-14T07:09:02.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936024+0000", 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7693000000000002e-05 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936037+0000", 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3303e-05 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956267+0000", 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020229626000000001 2024-09-14T07:09:02.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956482+0000", 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000215069 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956704+0000", 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022182500000000001 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.651 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.651 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:02.652 INFO:teuthology.orchestra.run.smithi145.stdout:2461 read 0x1abd800 thru 0x1ac45ff (0x6e00 bytes) 2024-09-14T07:09:02.652 INFO:teuthology.orchestra.run.smithi145.stdout:2464 write 0xd64f600 thru 0xd65589f (0x62a0 bytes) 2024-09-14T07:09:02.652 INFO:teuthology.orchestra.run.smithi145.stdout:2465 trunc from 0xd655a00 to 0xd6c4200 2024-09-14T07:09:02.653 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:02.653 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:02.648+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:02.660 INFO:teuthology.orchestra.run.smithi145.stdout:2466 write 0x6871c00 thru 0x687a1d4 (0x85d5 bytes) 2024-09-14T07:09:02.662 INFO:teuthology.orchestra.run.smithi145.stdout:2467 punch from 0xc741e00 to 0xc74c400, (0xa600 bytes) 2024-09-14T07:09:02.663 INFO:teuthology.orchestra.run.smithi145.stdout:2470 write 0x8448600 thru 0x8453cee (0xb6ef bytes) 2024-09-14T07:09:02.663 INFO:teuthology.orchestra.run.smithi145.stdout:2471 read 0xa4c6400 thru 0xa4c7bd7 (0x17d8 bytes) 2024-09-14T07:09:02.671 INFO:teuthology.orchestra.run.smithi145.stdout:2473 punch from 0xcebf800 to 0xcec8200, (0x8a00 bytes) 2024-09-14T07:09:02.672 INFO:teuthology.orchestra.run.smithi145.stdout:2477 write 0x8668800 thru 0x866fdc2 (0x75c3 bytes) 2024-09-14T07:09:02.673 INFO:teuthology.orchestra.run.smithi145.stdout:2482 punch from 0xa53aa00 to 0xa53b800, (0xe00 bytes) 2024-09-14T07:09:02.688 INFO:teuthology.orchestra.run.smithi145.stdout:2486 trunc from 0xd6c4200 to 0x5bdcc00 2024-09-14T07:09:02.753 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:02.748+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:02.753 INFO:teuthology.orchestra.run.smithi145.stdout:2489 write 0x7c84800 thru 0x7c8c537 (0x7d38 bytes) 2024-09-14T07:09:02.763 INFO:teuthology.orchestra.run.smithi145.stdout:2492 trunc from 0x7c8c600 to 0xc8a0c00 2024-09-14T07:09:02.783 INFO:teuthology.orchestra.run.smithi145.stdout:2494 read 0x7b79a00 thru 0x7b7b895 (0x1e96 bytes) 2024-09-14T07:09:02.805 INFO:teuthology.orchestra.run.smithi145.stdout:2495 write 0xcb45400 thru 0xcb4bc69 (0x686a bytes) 2024-09-14T07:09:02.817 INFO:teuthology.orchestra.run.smithi145.stdout:2498 trunc from 0xcb4be00 to 0x6d70800 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5062.0:1360 3.1a e79/57 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:head v 79'136, mlcod=79'134)", 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:02.848061+0000", 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00060036300000000003, 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063378800000000004, 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:02.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848061+0000", 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848061+0000", 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:02.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848062+0000", 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9199999999999999e-07 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848069+0000", 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2520000000000002e-06 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848086+0000", 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7232e-05 2024-09-14T07:09:02.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848098+0000", 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1429e-05 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848170+0000", 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2191000000000001e-05 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:02.848191+0000", 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0988999999999998e-05 2024-09-14T07:09:02.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:02.852 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:09:02.852 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:02.853 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:02.848+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:02.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:02.869 INFO:teuthology.orchestra.run.smithi145.stdout:2501 punch from 0x5f6fc00 to 0x5f75c00, (0x6000 bytes) 2024-09-14T07:09:02.900 INFO:teuthology.orchestra.run.smithi145.stdout:2502 write 0x8e3ce00 thru 0x8e40034 (0x3235 bytes) 2024-09-14T07:09:02.903 INFO:teuthology.orchestra.run.smithi145.stdout:2503 read 0x4675c00 thru 0x467afa8 (0x53a9 bytes) 2024-09-14T07:09:02.905 INFO:teuthology.orchestra.run.smithi145.stdout:2505 punch from 0x9800 to 0xf400, (0x5c00 bytes) 2024-09-14T07:09:02.905 INFO:teuthology.orchestra.run.smithi145.stdout:2506 punch from 0x2428800 to 0x2429800, (0x1000 bytes) 2024-09-14T07:09:02.937 INFO:teuthology.orchestra.run.smithi145.stdout:2511 trunc from 0x8e40200 to 0x1246a00 2024-09-14T07:09:02.954 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:02.948+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:03.009 INFO:teuthology.orchestra.run.smithi145.stdout:2512 write 0x9ae3000 thru 0x9aee594 (0xb595 bytes) 2024-09-14T07:09:03.032 INFO:teuthology.orchestra.run.smithi145.stdout:2514 trunc from 0x9aee600 to 0x17c4600 2024-09-14T07:09:03.036 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:03.036 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:03.036 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:03.036 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:03.036 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:03.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:03.054 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:03.048+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:03.107 INFO:teuthology.orchestra.run.smithi145.stdout:2516 trunc from 0x17c4600 to 0xeb6c000 2024-09-14T07:09:03.123 INFO:teuthology.orchestra.run.smithi145.stdout:2518 write 0x9a92600 thru 0x9a9f85b (0xd25c bytes) 2024-09-14T07:09:03.126 INFO:teuthology.orchestra.run.smithi145.stdout:2519 write 0x6064200 thru 0x606e523 (0xa324 bytes) 2024-09-14T07:09:03.127 INFO:teuthology.orchestra.run.smithi145.stdout:2523 write 0xb7e3800 thru 0xb7ec105 (0x8906 bytes) 2024-09-14T07:09:03.128 INFO:teuthology.orchestra.run.smithi145.stdout:2526 punch from 0xb2b5000 to 0xb2be600, (0x9600 bytes) 2024-09-14T07:09:03.129 INFO:teuthology.orchestra.run.smithi145.stdout:2528 read 0x7898c00 thru 0x78a330e (0xa70f bytes) 2024-09-14T07:09:03.139 INFO:teuthology.orchestra.run.smithi145.stdout:2531 write 0x694e800 thru 0x694ea1e (0x21f bytes) 2024-09-14T07:09:03.140 INFO:teuthology.orchestra.run.smithi145.stdout:2533 write 0x16c1200 thru 0x16cd284 (0xc085 bytes) 2024-09-14T07:09:03.151 INFO:teuthology.orchestra.run.smithi145.stdout:2536 trunc from 0xeb6c000 to 0xa4a0c00 2024-09-14T07:09:03.156 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:03.152+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:03.190 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:03.190 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:03.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:03.190 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1904 3.b 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9175040~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.900422063999997, 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.101088252, 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1904 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288894+0000", 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.666e-06 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315149+0000", 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026254493 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315177+0000", 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315200+0000", 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2949e-05 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315307+0000", 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107222 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315483+0000", 2024-09-14T07:09:03.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017589 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.317437+0000", 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019539890000000002 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.384167+0000", 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066730085999999994 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389856+0000", 2024-09-14T07:09:03.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056887079999999998 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389925+0000", 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8977e-05 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389981+0000", 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6158e-05 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.509529696000001, 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:03.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4071 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18219008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892767+0000", 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2965480179999993, 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080854407000000003, 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4071 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892767+0000", 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892767+0000", 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892768+0000", 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.032e-06 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892775+0000", 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9330000000000004e-06 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892795+0000", 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0338000000000001e-05 2024-09-14T07:09:03.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892808+0000", 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2717e-05 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973302+0000", 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080493947999999996 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973456+0000", 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015434199999999999 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973621+0000", 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016509700000000001 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4072 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18513920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892826+0000", 2024-09-14T07:09:03.205 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2964884810000008, 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081162628000000001, 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4072 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892826+0000", 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892826+0000", 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892827+0000", 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0750000000000001e-06 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892832+0000", 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7539999999999997e-06 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892850+0000", 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8077e-05 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892864+0000", 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4385e-05 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973672+0000", 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080807244 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973812+0000", 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014033700000000001 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973989+0000", 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176756 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4073 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18808832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892901+0000", 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2964135579999994, 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081457293, 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4073 2024-09-14T07:09:03.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892901+0000", 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892901+0000", 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892902+0000", 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.094e-06 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892910+0000", 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9899999999999997e-06 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892934+0000", 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3867999999999999e-05 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892947+0000", 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3096e-05 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974040+0000", 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081092483000000007 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974180+0000", 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014082999999999999 2024-09-14T07:09:03.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974358+0000", 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000177932 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4074 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18874368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892974+0000", 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2963407700000005, 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081755193000000004, 2024-09-14T07:09:03.214 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4074 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892974+0000", 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892974+0000", 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892975+0000", 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892981+0000", 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9349999999999999e-06 2024-09-14T07:09:03.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892997+0000", 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5996999999999999e-05 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893026+0000", 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9169000000000001e-05 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974408+0000", 2024-09-14T07:09:03.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081382198000000003 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974547+0000", 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013913 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974729+0000", 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018163899999999999 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4075 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19103744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.893057+0000", 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2962573420000005, 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082041696999999997, 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4075 2024-09-14T07:09:03.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893057+0000", 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893057+0000", 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893058+0000", 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0100000000000001e-06 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893064+0000", 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893082+0000", 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8827000000000001e-05 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893097+0000", 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.466e-05 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974778+0000", 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081680536999999998 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974915+0000", 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013752099999999999 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975099+0000", 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018398199999999999 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4076 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19169280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.893124+0000", 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2961909889999994, 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082343668999999994, 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4076 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893124+0000", 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893124+0000", 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893125+0000", 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.95e-07 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893131+0000", 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2230000000000004e-06 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893149+0000", 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7694e-05 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893163+0000", 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.468e-05 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975149+0000", 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081985303999999995 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975288+0000", 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013894999999999999 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975467+0000", 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017982299999999999 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4077 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19464192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.227 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.893182+0000", 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2961324269999999, 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082699184999999995, 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4077 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893182+0000", 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893182+0000", 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893183+0000", 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1650000000000001e-06 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893188+0000", 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.549e-06 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893204+0000", 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5534999999999999e-05 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893220+0000", 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6617e-05 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975569+0000", 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082349011999999999 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975708+0000", 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013850599999999999 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975881+0000", 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017380100000000001 2024-09-14T07:09:03.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4195 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17285120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899732+0000", 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2895828159999994, 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080966627999999999, 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4195 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899732+0000", 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899732+0000", 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899733+0000", 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6700000000000002e-07 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899736+0000", 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9059999999999998e-06 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899743+0000", 2024-09-14T07:09:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5220000000000001e-06 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899752+0000", 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0219999999999993e-06 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980372+0000", 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080620118000000004 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980531+0000", 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015877299999999999 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980699+0000", 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016741999999999999 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4196 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17350656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.236 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899761+0000", 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2895532010000004, 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081365215000000005, 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4196 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899761+0000", 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899761+0000", 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899762+0000", 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7199999999999999e-07 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899765+0000", 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6120000000000001e-06 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899772+0000", 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2849999999999997e-06 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899778+0000", 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980805+0000", 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081027063999999996 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980949+0000", 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014400799999999999 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981127+0000", 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017771599999999999 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.240 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4197 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17645568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2895291289999999, 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081746397999999998, 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4197 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:03.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899789+0000", 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7350000000000001e-06 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899796+0000", 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1180000000000003e-06 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899802+0000", 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9170000000000003e-06 2024-09-14T07:09:03.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981211+0000", 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081408736999999995 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981364+0000", 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015300700000000001 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981532+0000", 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016837199999999999 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.245 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4198 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17940480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899857+0000", 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2894579460000006, 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082043290000000005, 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4198 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899857+0000", 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899857+0000", 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899858+0000", 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0500000000000002e-07 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899862+0000", 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2490000000000003e-06 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899874+0000", 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1603000000000001e-05 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899881+0000", 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3300000000000001e-06 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981583+0000", 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081701766999999995 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981716+0000", 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013313100000000001 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981900+0000", 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000184305 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.249 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4199 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18235392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899975+0000", 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2893394140000005, 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082337240000000006, 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4199 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899975+0000", 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899975+0000", 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899977+0000", 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.638e-06 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899984+0000", 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5440000000000001e-06 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900041+0000", 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6910000000000002e-05 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900060+0000", 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8159e-05 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982003+0000", 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081943960999999996 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982142+0000", 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013828899999999999 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982313+0000", 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017073899999999999 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4200 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18530304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900204+0000", 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2891107339999994, 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082508573000000002, 2024-09-14T07:09:03.254 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4200 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900204+0000", 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900204+0000", 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900206+0000", 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7710000000000001e-06 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900214+0000", 2024-09-14T07:09:03.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.456e-06 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900235+0000", 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-05 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900270+0000", 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4581e-05 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982396+0000", 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082126331999999996 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982547+0000", 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000151237 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982713+0000", 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526600000000001 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4201 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18825216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900295+0000", 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2890196960000004, 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082796755, 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4201 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.259 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900295+0000", 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900295+0000", 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900296+0000", 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1400000000000001e-06 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900303+0000", 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3770000000000001e-06 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900319+0000", 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6657e-05 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900340+0000", 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1118999999999999e-05 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982803+0000", 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082462368999999994 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982929+0000", 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126823 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983092+0000", 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016227 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4202 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18890752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900364+0000", 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.288950883, 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083126310999999994, 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4202 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900364+0000", 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900364+0000", 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900365+0000", 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.04e-06 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900370+0000", 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.045e-06 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900406+0000", 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5914000000000002e-05 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900420+0000", 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4012999999999999e-05 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983174+0000", 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082754105999999994 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983326+0000", 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015226000000000001 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983490+0000", 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000163933 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.267 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4203 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19120128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900469+0000", 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2888457100000004, 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083390407, 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4203 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900469+0000", 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900469+0000", 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900471+0000", 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.832e-06 2024-09-14T07:09:03.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900477+0000", 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900498+0000", 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0307000000000001e-05 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900511+0000", 2024-09-14T07:09:03.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3298999999999999e-05 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983579+0000", 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083067618999999995 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983704+0000", 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012488899999999999 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983859+0000", 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015579299999999999 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4204 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19185664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900544+0000", 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2887709889999996, 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083738039, 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.272 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4204 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900544+0000", 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900544+0000", 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900545+0000", 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0640000000000001e-06 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900552+0000", 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9029999999999998e-06 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900574+0000", 2024-09-14T07:09:03.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1974e-05 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900587+0000", 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3492999999999999e-05 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983941+0000", 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083354182999999998 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984095+0000", 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015380600000000001 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984282+0000", 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018661599999999999 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4205 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19480576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900631+0000", 2024-09-14T07:09:03.276 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2886836929999994, 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.084031554999999994, 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4205 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900631+0000", 2024-09-14T07:09:03.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900631+0000", 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900632+0000", 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4900000000000005e-07 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:03.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900636+0000", 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1030000000000004e-06 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900653+0000", 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6671999999999999e-05 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900660+0000", 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3819999999999998e-06 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984370+0000", 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083710190000000004 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984497+0000", 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012703399999999999 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984663+0000", 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016532499999999999 2024-09-14T07:09:03.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:03.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:03.281 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:03.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:03.284 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:03.276+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2537 write 0xed88400 thru 0xed8d1d4 (0x4dd5 bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2538 write 0x3dde800 thru 0x3de3c00 (0x5401 bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2540 punch from 0x5da5600 to 0x5db5200, (0xfc00 bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2544 punch from 0x829d600 to 0x82a5400, (0x7e00 bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2546 read 0xb043a00 thru 0xb04ac9c (0x729d bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2547 punch from 0xa56e800 to 0xa579200, (0xaa00 bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2549 write 0xc846800 thru 0xc84b18d (0x498e bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2555 read 0xd6c1e00 thru 0xd6cc5fe (0xa7ff bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2557 read 0x4a9ee00 thru 0x4aaa79a (0xb99b bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2558 write 0x6ee7800 thru 0x6ef4dd7 (0xd5d8 bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2559 trunc from 0xed8d200 to 0xb018200 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2562 read 0x3115400 thru 0x3123a71 (0xe672 bytes) 2024-09-14T07:09:03.285 INFO:teuthology.orchestra.run.smithi145.stdout:2563 write 0x5e29400 thru 0x5e2aa3d (0x163e bytes) 2024-09-14T07:09:03.286 INFO:teuthology.orchestra.run.smithi145.stdout:2564 punch from 0x3c42c00 to 0x3c4ae00, (0x8200 bytes) 2024-09-14T07:09:03.286 INFO:teuthology.orchestra.run.smithi145.stdout:2565 punch from 0x6408e00 to 0x6411600, (0x8800 bytes) 2024-09-14T07:09:03.286 INFO:teuthology.orchestra.run.smithi145.stdout:2569 read 0x11e4800 thru 0x11f4157 (0xf958 bytes) 2024-09-14T07:09:03.286 INFO:teuthology.orchestra.run.smithi145.stdout:2570 write 0x1541800 thru 0x154d4fa (0xbcfb bytes) 2024-09-14T07:09:03.287 INFO:teuthology.orchestra.run.smithi145.stdout:2572 read 0x1c02200 thru 0x1c07068 (0x4e69 bytes) 2024-09-14T07:09:03.290 INFO:teuthology.orchestra.run.smithi145.stdout:2574 read 0x8394400 thru 0x8395ee1 (0x1ae2 bytes) 2024-09-14T07:09:03.291 INFO:teuthology.orchestra.run.smithi145.stdout:2575 write 0x75f5000 thru 0x75fdba1 (0x8ba2 bytes) 2024-09-14T07:09:03.293 INFO:teuthology.orchestra.run.smithi145.stdout:2577 punch from 0x87cb200 to 0x87d6400, (0xb200 bytes) 2024-09-14T07:09:03.293 INFO:teuthology.orchestra.run.smithi145.stdout:2578 read 0x1d7ac00 thru 0x1d7da79 (0x2e7a bytes) 2024-09-14T07:09:03.295 INFO:teuthology.orchestra.run.smithi145.stdout:2579 write 0x27c0c00 thru 0x27cd9fa (0xcdfb bytes) 2024-09-14T07:09:03.300 INFO:teuthology.orchestra.run.smithi145.stdout:2581 punch from 0x5b45a00 to 0x5b50400, (0xaa00 bytes) 2024-09-14T07:09:03.301 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T07:09:03.301 INFO:teuthology.orchestra.run.smithi145.stdout:2582 punch from 0x6fc8600 to 0x6fd5c00, (0xd600 bytes) 2024-09-14T07:09:03.301 INFO:teuthology.orchestra.run.smithi145.stdout:2583 read 0x9a24a00 thru 0x9a29fb9 (0x55ba bytes) 2024-09-14T07:09:03.304 INFO:teuthology.orchestra.run.smithi145.stdout:2584 write 0x55f3400 thru 0x55f93b8 (0x5fb9 bytes) 2024-09-14T07:09:03.305 INFO:teuthology.orchestra.run.smithi145.stdout:2587 read 0x19a3a00 thru 0x19ab266 (0x7867 bytes) 2024-09-14T07:09:03.309 INFO:teuthology.orchestra.run.smithi145.stdout:2588 trunc from 0xb018200 to 0x9a8e600 2024-09-14T07:09:03.328 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T07:09:03.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:03.344 INFO:teuthology.orchestra.run.smithi145.stdout:2590 read 0x5a4cc00 thru 0x5a5bd25 (0xf126 bytes) 2024-09-14T07:09:03.347 INFO:teuthology.orchestra.run.smithi145.stdout:2591 read 0x4c51200 thru 0x4c5a239 (0x903a bytes) 2024-09-14T07:09:03.349 INFO:teuthology.orchestra.run.smithi145.stdout:2593 clone 18 order 18 su 2048 sc 11 2024-09-14T07:09:03.383 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:03.380+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:03.433 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:03.434 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:03.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:03.484 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:03.484+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:03.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:03.574 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:03.574 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:03.586 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:03.584+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:03.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:03.679 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:03.687 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:03.684+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:03.691 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:03.699 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-14T07:09:03.295809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1097584999,"num_objects":510,"num_object_clones":76,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":164,"num_read":83793,"num_read_kb":425881,"num_write":6696,"num_write_kb":180050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":634,"num_bytes_recovered":79843423,"num_keys_recovered":209,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5362,"ondisk_log_size":5362,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":226288,"kb_used_data":24564,"kb_used_omap":7,"kb_used_meta":201592,"kb_avail":471632912,"statfs":{"total":483183820800,"available":482952101888,"internally_reserved":0,"allocated":25153536,"data_stored":243920172,"data_compressed":6717964,"data_compressed_allocated":17309696,"data_compressed_original":240127488,"omap_allocated":7973,"internal_metadata":206430427},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":75308838,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":32,"num_read":12930,"num_read_kb":66325,"num_write":1185,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031122"},"pg_stats":[{"pgid":"3.1f","version":"76'95","reported_seq":2515,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271117+0000","last_change":"2024-09-14T07:08:49.496718+0000","last_active":"2024-09-14T07:09:00.271117+0000","last_peered":"2024-09-14T07:09:00.271117+0000","last_clean":"2024-09-14T07:09:00.271117+0000","last_became_active":"2024-09-14T07:08:49.217745+0000","last_became_peered":"2024-09-14T07:08:49.217745+0000","last_unstale":"2024-09-14T07:09:00.271117+0000","last_undegraded":"2024-09-14T07:09:00.271117+0000","last_fullsized":"2024-09-14T07:09:00.271117+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:11:45.435715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":55218192,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":2287,"num_read_kb":37252,"num_write":171,"num_write_kb":9596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":7794192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'317","reported_seq":4458,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271593+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:00.271593+0000","last_peered":"2024-09-14T07:09:00.271593+0000","last_clean":"2024-09-14T07:09:00.271593+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:00.271593+0000","last_undegraded":"2024-09-14T07:09:00.271593+0000","last_fullsized":"2024-09-14T07:09:00.271593+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:32.148218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":48997615,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":5154,"num_read_kb":33370,"num_write":538,"num_write_kb":11173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"76'233","reported_seq":4055,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267554+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:00.267554+0000","last_peered":"2024-09-14T07:09:00.267554+0000","last_clean":"2024-09-14T07:09:00.267554+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:00.267554+0000","last_undegraded":"2024-09-14T07:09:00.267554+0000","last_fullsized":"2024-09-14T07:09:00.267554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:19:04.925803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":8212480,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":5096,"num_read_kb":10852,"num_write":133,"num_write_kb":1835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"76'80","reported_seq":1296,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267494+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:00.267494+0000","last_peered":"2024-09-14T07:09:00.267494+0000","last_clean":"2024-09-14T07:09:00.267494+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:00.267494+0000","last_undegraded":"2024-09-14T07:09:00.267494+0000","last_fullsized":"2024-09-14T07:09:00.267494+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:03:04.642439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":15138852,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1052,"num_read_kb":5319,"num_write":78,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"78'313","reported_seq":2854,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.183088+0000","last_change":"2024-09-14T07:08:52.243357+0000","last_active":"2024-09-14T07:09:02.183088+0000","last_peered":"2024-09-14T07:09:02.183088+0000","last_clean":"2024-09-14T07:09:02.183088+0000","last_became_active":"2024-09-14T07:08:52.241415+0000","last_became_peered":"2024-09-14T07:08:52.241415+0000","last_unstale":"2024-09-14T07:09:02.183088+0000","last_undegraded":"2024-09-14T07:09:02.183088+0000","last_fullsized":"2024-09-14T07:09:02.183088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:33.731123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":94706363,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":2991,"num_read_kb":8831,"num_write":440,"num_write_kb":8661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4094396,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"78'129","reported_seq":3419,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.273996+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:02.273996+0000","last_peered":"2024-09-14T07:09:02.273996+0000","last_clean":"2024-09-14T07:09:02.273996+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:02.273996+0000","last_undegraded":"2024-09-14T07:09:02.273996+0000","last_fullsized":"2024-09-14T07:09:02.273996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:54:57.601863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":69468295,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":4159,"num_read_kb":7826,"num_write":167,"num_write_kb":9312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"76'104","reported_seq":1439,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267622+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:00.267622+0000","last_peered":"2024-09-14T07:09:00.267622+0000","last_clean":"2024-09-14T07:09:00.267622+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:00.267622+0000","last_undegraded":"2024-09-14T07:09:00.267622+0000","last_fullsized":"2024-09-14T07:09:00.267622+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:04:19.309443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":21731419,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":1309,"num_read_kb":8735,"num_write":121,"num_write_kb":2625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"76'127","reported_seq":1572,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271267+0000","last_change":"2024-09-14T07:08:52.234551+0000","last_active":"2024-09-14T07:09:00.271267+0000","last_peered":"2024-09-14T07:09:00.271267+0000","last_clean":"2024-09-14T07:09:00.271267+0000","last_became_active":"2024-09-14T07:08:52.234002+0000","last_became_peered":"2024-09-14T07:08:52.234002+0000","last_unstale":"2024-09-14T07:09:00.271267+0000","last_undegraded":"2024-09-14T07:09:00.271267+0000","last_fullsized":"2024-09-14T07:09:00.271267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:52:57.941656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":48335920,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1373,"num_read_kb":4835,"num_write":250,"num_write_kb":13197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1118768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"74'62","reported_seq":646,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267592+0000","last_change":"2024-09-14T07:08:49.269903+0000","last_active":"2024-09-14T07:09:00.267592+0000","last_peered":"2024-09-14T07:09:00.267592+0000","last_clean":"2024-09-14T07:09:00.267592+0000","last_became_active":"2024-09-14T07:08:49.218168+0000","last_became_peered":"2024-09-14T07:08:49.218168+0000","last_unstale":"2024-09-14T07:09:00.267592+0000","last_undegraded":"2024-09-14T07:09:00.267592+0000","last_fullsized":"2024-09-14T07:09:00.267592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:53:21.570579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":4509734,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":387,"num_read_kb":4056,"num_write":70,"num_write_kb":2010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":503846,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"76'141","reported_seq":2417,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267579+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:00.267579+0000","last_peered":"2024-09-14T07:09:00.267579+0000","last_clean":"2024-09-14T07:09:00.267579+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:00.267579+0000","last_undegraded":"2024-09-14T07:09:00.267579+0000","last_fullsized":"2024-09-14T07:09:00.267579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:21:34.044177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":33319974,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":2326,"num_read_kb":20290,"num_write":140,"num_write_kb":3692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"76'126","reported_seq":2791,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271048+0000","last_change":"2024-09-14T07:08:49.770980+0000","last_active":"2024-09-14T07:09:00.271048+0000","last_peered":"2024-09-14T07:09:00.271048+0000","last_clean":"2024-09-14T07:09:00.271048+0000","last_became_active":"2024-09-14T07:08:49.213956+0000","last_became_peered":"2024-09-14T07:08:49.213956+0000","last_unstale":"2024-09-14T07:09:00.271048+0000","last_undegraded":"2024-09-14T07:09:00.271048+0000","last_fullsized":"2024-09-14T07:09:00.271048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:07:11.974717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":62939185,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":2806,"num_read_kb":34536,"num_write":210,"num_write_kb":12544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":8864817,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"76'146","reported_seq":4527,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271032+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:00.271032+0000","last_peered":"2024-09-14T07:09:00.271032+0000","last_clean":"2024-09-14T07:09:00.271032+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:00.271032+0000","last_undegraded":"2024-09-14T07:09:00.271032+0000","last_fullsized":"2024-09-14T07:09:00.271032+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:49:23.542131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":53464080,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":4649,"num_read_kb":18460,"num_write":117,"num_write_kb":3299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271254+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:00.271254+0000","last_peered":"2024-09-14T07:09:00.271254+0000","last_clean":"2024-09-14T07:09:00.271254+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:00.271254+0000","last_undegraded":"2024-09-14T07:09:00.271254+0000","last_fullsized":"2024-09-14T07:09:00.271254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"76'264","reported_seq":1435,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267780+0000","last_change":"2024-09-14T07:08:52.239987+0000","last_active":"2024-09-14T07:09:00.267780+0000","last_peered":"2024-09-14T07:09:00.267780+0000","last_clean":"2024-09-14T07:09:00.267780+0000","last_became_active":"2024-09-14T07:08:52.239406+0000","last_became_peered":"2024-09-14T07:08:52.239406+0000","last_unstale":"2024-09-14T07:09:00.267780+0000","last_undegraded":"2024-09-14T07:09:00.267780+0000","last_fullsized":"2024-09-14T07:09:00.267780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:00:33.900361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":397545,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":2668,"num_read_kb":2334,"num_write":290,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":239,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.269139+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:00.269139+0000","last_peered":"2024-09-14T07:09:00.269139+0000","last_clean":"2024-09-14T07:09:00.269139+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:00.269139+0000","last_undegraded":"2024-09-14T07:09:00.269139+0000","last_fullsized":"2024-09-14T07:09:00.269139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'162","reported_seq":1353,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271687+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:00.271687+0000","last_peered":"2024-09-14T07:09:00.271687+0000","last_clean":"2024-09-14T07:09:00.271687+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:00.271687+0000","last_undegraded":"2024-09-14T07:09:00.271687+0000","last_fullsized":"2024-09-14T07:09:00.271687+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:18:23.853525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":20203550,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1795,"num_read_kb":7488,"num_write":199,"num_write_kb":2043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271205+0000","last_change":"2024-09-14T07:08:49.212698+0000","last_active":"2024-09-14T07:09:00.271205+0000","last_peered":"2024-09-14T07:09:00.271205+0000","last_clean":"2024-09-14T07:09:00.271205+0000","last_became_active":"2024-09-14T07:08:49.211419+0000","last_became_peered":"2024-09-14T07:08:49.211419+0000","last_unstale":"2024-09-14T07:09:00.271205+0000","last_undegraded":"2024-09-14T07:09:00.271205+0000","last_fullsized":"2024-09-14T07:09:00.271205+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:07:35.929274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"76'415","reported_seq":2033,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267706+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:00.267706+0000","last_peered":"2024-09-14T07:09:00.267706+0000","last_clean":"2024-09-14T07:09:00.267706+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:00.267706+0000","last_undegraded":"2024-09-14T07:09:00.267706+0000","last_fullsized":"2024-09-14T07:09:00.267706+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:18:20.981962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":393271,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":3847,"num_read_kb":3595,"num_write":617,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267754+0000","last_change":"2024-09-14T07:08:49.213968+0000","last_active":"2024-09-14T07:09:00.267754+0000","last_peered":"2024-09-14T07:09:00.267754+0000","last_clean":"2024-09-14T07:09:00.267754+0000","last_became_active":"2024-09-14T07:08:49.213320+0000","last_became_peered":"2024-09-14T07:08:49.213320+0000","last_unstale":"2024-09-14T07:09:00.267754+0000","last_undegraded":"2024-09-14T07:09:00.267754+0000","last_fullsized":"2024-09-14T07:09:00.267754+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:35.337724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'45","reported_seq":1780,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267698+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:00.267698+0000","last_peered":"2024-09-14T07:09:00.267698+0000","last_clean":"2024-09-14T07:09:00.267698+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:00.267698+0000","last_undegraded":"2024-09-14T07:09:00.267698+0000","last_fullsized":"2024-09-14T07:09:00.267698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:30.644693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":8269856,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":1520,"num_read_kb":11887,"num_write":55,"num_write_kb":3208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"78'46","reported_seq":733,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.257924+0000","last_change":"2024-09-14T07:08:49.426259+0000","last_active":"2024-09-14T07:09:02.257924+0000","last_peered":"2024-09-14T07:09:02.257924+0000","last_clean":"2024-09-14T07:09:02.257924+0000","last_became_active":"2024-09-14T07:08:49.210033+0000","last_became_peered":"2024-09-14T07:08:49.210033+0000","last_unstale":"2024-09-14T07:09:02.257924+0000","last_undegraded":"2024-09-14T07:09:02.257924+0000","last_fullsized":"2024-09-14T07:09:02.257924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:15:53.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":44498944,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":683,"num_read_kb":7221,"num_write":119,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":19087872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267636+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:00.267636+0000","last_peered":"2024-09-14T07:09:00.267636+0000","last_clean":"2024-09-14T07:09:00.267636+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:00.267636+0000","last_undegraded":"2024-09-14T07:09:00.267636+0000","last_fullsized":"2024-09-14T07:09:00.267636+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"76'143","reported_seq":1724,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267706+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:00.267706+0000","last_peered":"2024-09-14T07:09:00.267706+0000","last_clean":"2024-09-14T07:09:00.267706+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:00.267706+0000","last_undegraded":"2024-09-14T07:09:00.267706+0000","last_fullsized":"2024-09-14T07:09:00.267706+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:24.601129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":3164834,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":2654,"num_read_kb":5086,"num_write":98,"num_write_kb":1367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271463+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:00.271463+0000","last_peered":"2024-09-14T07:09:00.271463+0000","last_clean":"2024-09-14T07:09:00.271463+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:00.271463+0000","last_undegraded":"2024-09-14T07:09:00.271463+0000","last_fullsized":"2024-09-14T07:09:00.271463+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"78'93","reported_seq":553,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.171547+0000","last_change":"2024-09-14T07:08:49.550839+0000","last_active":"2024-09-14T07:09:02.171547+0000","last_peered":"2024-09-14T07:09:02.171547+0000","last_clean":"2024-09-14T07:09:02.171547+0000","last_became_active":"2024-09-14T07:08:49.220855+0000","last_became_peered":"2024-09-14T07:08:49.220855+0000","last_unstale":"2024-09-14T07:09:02.171547+0000","last_undegraded":"2024-09-14T07:09:02.171547+0000","last_fullsized":"2024-09-14T07:09:02.171547+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:25:29.549402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":13242921,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":531,"num_read_kb":1731,"num_write":94,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":128082,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270475+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:00.270475+0000","last_peered":"2024-09-14T07:09:00.270475+0000","last_clean":"2024-09-14T07:09:00.270475+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:00.270475+0000","last_undegraded":"2024-09-14T07:09:00.270475+0000","last_fullsized":"2024-09-14T07:09:00.270475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"76'133","reported_seq":1487,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.269176+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:00.269176+0000","last_peered":"2024-09-14T07:09:00.269176+0000","last_clean":"2024-09-14T07:09:00.269176+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:00.269176+0000","last_undegraded":"2024-09-14T07:09:00.269176+0000","last_fullsized":"2024-09-14T07:09:00.269176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:40.510138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":42720288,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":1254,"num_read_kb":3239,"num_write":115,"num_write_kb":1256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":157,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267488+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:00.267488+0000","last_peered":"2024-09-14T07:09:00.267488+0000","last_clean":"2024-09-14T07:09:00.267488+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:00.267488+0000","last_undegraded":"2024-09-14T07:09:00.267488+0000","last_fullsized":"2024-09-14T07:09:00.267488+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":239,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270579+0000","last_change":"2024-09-14T07:08:49.338418+0000","last_active":"2024-09-14T07:09:00.270579+0000","last_peered":"2024-09-14T07:09:00.270579+0000","last_clean":"2024-09-14T07:09:00.270579+0000","last_became_active":"2024-09-14T07:08:49.217675+0000","last_became_peered":"2024-09-14T07:08:49.217675+0000","last_unstale":"2024-09-14T07:09:00.270579+0000","last_undegraded":"2024-09-14T07:09:00.270579+0000","last_fullsized":"2024-09-14T07:09:00.270579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.519350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"74'186","reported_seq":6412,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271235+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:00.271235+0000","last_peered":"2024-09-14T07:09:00.271235+0000","last_clean":"2024-09-14T07:09:00.271235+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:00.271235+0000","last_undegraded":"2024-09-14T07:09:00.271235+0000","last_fullsized":"2024-09-14T07:09:00.271235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:00:50.807270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":113640500,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":6020,"num_read_kb":52248,"num_write":337,"num_write_kb":22649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"78'338","reported_seq":3384,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.236785+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:02.236785+0000","last_peered":"2024-09-14T07:09:02.236785+0000","last_clean":"2024-09-14T07:09:02.236785+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:02.236785+0000","last_undegraded":"2024-09-14T07:09:02.236785+0000","last_fullsized":"2024-09-14T07:09:02.236785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:07:14.427467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":62256396,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":4598,"num_read_kb":22890,"num_write":389,"num_write_kb":5452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"76'70","reported_seq":469,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270943+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:00.270943+0000","last_peered":"2024-09-14T07:09:00.270943+0000","last_clean":"2024-09-14T07:09:00.270943+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:00.270943+0000","last_undegraded":"2024-09-14T07:09:00.270943+0000","last_fullsized":"2024-09-14T07:09:00.270943+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:11:18.610572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":2156560,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":240,"num_read_kb":1740,"num_write":77,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"77'435","reported_seq":4639,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267919+0000","last_change":"2024-09-14T07:08:52.240372+0000","last_active":"2024-09-14T07:09:00.267919+0000","last_peered":"2024-09-14T07:09:00.267919+0000","last_clean":"2024-09-14T07:09:00.267919+0000","last_became_active":"2024-09-14T07:08:52.239477+0000","last_became_peered":"2024-09-14T07:08:52.239477+0000","last_unstale":"2024-09-14T07:09:00.267919+0000","last_undegraded":"2024-09-14T07:09:00.267919+0000","last_fullsized":"2024-09-14T07:09:00.267919+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:35:19.679463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":43412716,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":6289,"num_read_kb":26177,"num_write":611,"num_write_kb":10494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8521964,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"76'166","reported_seq":2866,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267640+0000","last_change":"2024-09-14T07:08:52.240897+0000","last_active":"2024-09-14T07:09:00.267640+0000","last_peered":"2024-09-14T07:09:00.267640+0000","last_clean":"2024-09-14T07:09:00.267640+0000","last_became_active":"2024-09-14T07:08:52.239622+0000","last_became_peered":"2024-09-14T07:08:52.239622+0000","last_unstale":"2024-09-14T07:09:00.267640+0000","last_undegraded":"2024-09-14T07:09:00.267640+0000","last_fullsized":"2024-09-14T07:09:00.267640+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:23:14.104906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":29643264,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":3810,"num_read_kb":13000,"num_write":118,"num_write_kb":2863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":476160,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"75'105","reported_seq":2371,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271080+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:00.271080+0000","last_peered":"2024-09-14T07:09:00.271080+0000","last_clean":"2024-09-14T07:09:00.271080+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:00.271080+0000","last_undegraded":"2024-09-14T07:09:00.271080+0000","last_fullsized":"2024-09-14T07:09:00.271080+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:39:24.368465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":21025792,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":2164,"num_read_kb":10714,"num_write":93,"num_write_kb":1970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"76'193","reported_seq":2098,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267832+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:00.267832+0000","last_peered":"2024-09-14T07:09:00.267832+0000","last_clean":"2024-09-14T07:09:00.267832+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:00.267832+0000","last_undegraded":"2024-09-14T07:09:00.267832+0000","last_fullsized":"2024-09-14T07:09:00.267832+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:26.927492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":18718775,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":2847,"num_read_kb":9986,"num_write":147,"num_write_kb":2786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"76'132","reported_seq":927,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267712+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:00.267712+0000","last_peered":"2024-09-14T07:09:00.267712+0000","last_clean":"2024-09-14T07:09:00.267712+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:00.267712+0000","last_undegraded":"2024-09-14T07:09:00.267712+0000","last_fullsized":"2024-09-14T07:09:00.267712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:30:23.788694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":24046172,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":964,"num_read_kb":5000,"num_write":270,"num_write_kb":13588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"75'93","reported_seq":907,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267481+0000","last_change":"2024-09-14T07:08:49.325783+0000","last_active":"2024-09-14T07:09:00.267481+0000","last_peered":"2024-09-14T07:09:00.267481+0000","last_clean":"2024-09-14T07:09:00.267481+0000","last_became_active":"2024-09-14T07:08:49.218113+0000","last_became_peered":"2024-09-14T07:08:49.218113+0000","last_unstale":"2024-09-14T07:09:00.267481+0000","last_undegraded":"2024-09-14T07:09:00.267481+0000","last_fullsized":"2024-09-14T07:09:00.267481+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:46:25.951257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":16117248,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":653,"num_read_kb":1355,"num_write":188,"num_write_kb":12452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":41984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"76'210","reported_seq":2733,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270506+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:00.270506+0000","last_peered":"2024-09-14T07:09:00.270506+0000","last_clean":"2024-09-14T07:09:00.270506+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:00.270506+0000","last_undegraded":"2024-09-14T07:09:00.270506+0000","last_fullsized":"2024-09-14T07:09:00.270506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:30.416532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":18952847,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":3274,"num_read_kb":4789,"num_write":152,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"76'67","reported_seq":1448,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270506+0000","last_change":"2024-09-14T07:08:49.288841+0000","last_active":"2024-09-14T07:09:00.270506+0000","last_peered":"2024-09-14T07:09:00.270506+0000","last_clean":"2024-09-14T07:09:00.270506+0000","last_became_active":"2024-09-14T07:08:49.217146+0000","last_became_peered":"2024-09-14T07:08:49.217146+0000","last_unstale":"2024-09-14T07:09:00.270506+0000","last_undegraded":"2024-09-14T07:09:00.270506+0000","last_fullsized":"2024-09-14T07:09:00.270506+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:12:16.864389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":34771456,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":1132,"num_read_kb":5990,"num_write":65,"num_write_kb":1079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":736768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"76'118","reported_seq":2021,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.268023+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:00.268023+0000","last_peered":"2024-09-14T07:09:00.268023+0000","last_clean":"2024-09-14T07:09:00.268023+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:00.268023+0000","last_undegraded":"2024-09-14T07:09:00.268023+0000","last_fullsized":"2024-09-14T07:09:00.268023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:12:37.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":63319568,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":3195,"num_read_kb":34994,"num_write":124,"num_write_kb":3416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1096994612,"num_objects":506,"num_object_clones":76,"num_object_copies":1012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":506,"num_whiteouts":164,"num_read":83727,"num_read_kb":425826,"num_write":6593,"num_write_kb":178420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":630,"num_bytes_recovered":79253055,"num_keys_recovered":209,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22605824,"data_stored":263952664,"data_compressed":7458876,"data_compressed_allocated":18964480,"data_compressed_original":261818368,"omap_allocated":0,"internal_metadata":0},"log_size":5287,"ondisk_log_size":5287,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":19008,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49800,"kb_used_data":6128,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94322040,"statfs":{"total":96636764160,"available":96585768960,"internally_reserved":0,"allocated":6275072,"data_stored":66670968,"data_compressed":1475410,"data_compressed_allocated":4481024,"data_compressed_original":65751552,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607574,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41996,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329844,"statfs":{"total":96636764160,"available":96593760256,"internally_reserved":0,"allocated":2547712,"data_stored":12741793,"data_compressed":668708,"data_compressed_allocated":1294336,"data_compressed_original":12179456,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45428,"kb_used_data":5924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326412,"statfs":{"total":96636764160,"available":96590245888,"internally_reserved":0,"allocated":6066176,"data_stored":60124279,"data_compressed":1812355,"data_compressed_allocated":4354048,"data_compressed_original":59248640,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607575,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44764,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327076,"statfs":{"total":96636764160,"available":96590925824,"internally_reserved":0,"allocated":5369856,"data_stored":50426604,"data_compressed":1590185,"data_compressed_allocated":3751936,"data_compressed_original":49639936,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607575,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44300,"kb_used_data":4780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327540,"statfs":{"total":96636764160,"available":96591400960,"internally_reserved":0,"allocated":4894720,"data_stored":53956528,"data_compressed":1171306,"data_compressed_allocated":3428352,"data_compressed_original":53307904,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":51539607575,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1437696,"data_stored":17352863,"data_compressed":558105,"data_compressed_allocated":1204224,"data_compressed_original":17242624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3969024,"data_stored":53015618,"data_compressed":1164414,"data_compressed_allocated":3391488,"data_compressed_original":52718080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4489216,"data_stored":50076081,"data_compressed":1590185,"data_compressed_allocated":3751936,"data_compressed_original":49639936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5128192,"data_stored":59183388,"data_compressed":1806297,"data_compressed_allocated":4317184,"data_compressed_original":58675200,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5918720,"data_stored":71933463,"data_compressed":1671167,"data_compressed_allocated":5005312,"data_compressed_original":71363072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1662976,"data_stored":12391251,"data_compressed":668708,"data_compressed_allocated":1294336,"data_compressed_original":12179456,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:03.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:03.709 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:03.727 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:03.749 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:03.749 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:03.753 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:03.753 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:03.769 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:03.770 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:03.770 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:03.789 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:03.815 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:03.812+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:03.888 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:03.884+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:03.902 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:03.902 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:03.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:03.989 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:03.984+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:04.010 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:04.028 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-14T07:09:03.295809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1097584999,"num_objects":510,"num_object_clones":76,"num_object_copies":1020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":510,"num_whiteouts":164,"num_read":83793,"num_read_kb":425881,"num_write":6696,"num_write_kb":180050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":634,"num_bytes_recovered":79843423,"num_keys_recovered":209,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5362,"ondisk_log_size":5362,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":226288,"kb_used_data":24564,"kb_used_omap":7,"kb_used_meta":201592,"kb_avail":471632912,"statfs":{"total":483183820800,"available":482952101888,"internally_reserved":0,"allocated":25153536,"data_stored":243920172,"data_compressed":6717964,"data_compressed_allocated":17309696,"data_compressed_original":240127488,"omap_allocated":7973,"internal_metadata":206430427},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":75308838,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":32,"num_read":12930,"num_read_kb":66325,"num_write":1185,"num_write_kb":8989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031122"},"pg_stats":[{"pgid":"3.1f","version":"76'95","reported_seq":2515,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271117+0000","last_change":"2024-09-14T07:08:49.496718+0000","last_active":"2024-09-14T07:09:00.271117+0000","last_peered":"2024-09-14T07:09:00.271117+0000","last_clean":"2024-09-14T07:09:00.271117+0000","last_became_active":"2024-09-14T07:08:49.217745+0000","last_became_peered":"2024-09-14T07:08:49.217745+0000","last_unstale":"2024-09-14T07:09:00.271117+0000","last_undegraded":"2024-09-14T07:09:00.271117+0000","last_fullsized":"2024-09-14T07:09:00.271117+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:11:45.435715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":55218192,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":2287,"num_read_kb":37252,"num_write":171,"num_write_kb":9596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":7794192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'317","reported_seq":4458,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271593+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:00.271593+0000","last_peered":"2024-09-14T07:09:00.271593+0000","last_clean":"2024-09-14T07:09:00.271593+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:00.271593+0000","last_undegraded":"2024-09-14T07:09:00.271593+0000","last_fullsized":"2024-09-14T07:09:00.271593+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:32.148218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":48997615,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":5154,"num_read_kb":33370,"num_write":538,"num_write_kb":11173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"76'233","reported_seq":4055,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267554+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:00.267554+0000","last_peered":"2024-09-14T07:09:00.267554+0000","last_clean":"2024-09-14T07:09:00.267554+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:00.267554+0000","last_undegraded":"2024-09-14T07:09:00.267554+0000","last_fullsized":"2024-09-14T07:09:00.267554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:19:04.925803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":8212480,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":5096,"num_read_kb":10852,"num_write":133,"num_write_kb":1835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"76'80","reported_seq":1296,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267494+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:00.267494+0000","last_peered":"2024-09-14T07:09:00.267494+0000","last_clean":"2024-09-14T07:09:00.267494+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:00.267494+0000","last_undegraded":"2024-09-14T07:09:00.267494+0000","last_fullsized":"2024-09-14T07:09:00.267494+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:03:04.642439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":15138852,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1052,"num_read_kb":5319,"num_write":78,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"78'313","reported_seq":2854,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.183088+0000","last_change":"2024-09-14T07:08:52.243357+0000","last_active":"2024-09-14T07:09:02.183088+0000","last_peered":"2024-09-14T07:09:02.183088+0000","last_clean":"2024-09-14T07:09:02.183088+0000","last_became_active":"2024-09-14T07:08:52.241415+0000","last_became_peered":"2024-09-14T07:08:52.241415+0000","last_unstale":"2024-09-14T07:09:02.183088+0000","last_undegraded":"2024-09-14T07:09:02.183088+0000","last_fullsized":"2024-09-14T07:09:02.183088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:33.731123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":94706363,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":2991,"num_read_kb":8831,"num_write":440,"num_write_kb":8661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":4094396,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"78'129","reported_seq":3419,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.273996+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:02.273996+0000","last_peered":"2024-09-14T07:09:02.273996+0000","last_clean":"2024-09-14T07:09:02.273996+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:02.273996+0000","last_undegraded":"2024-09-14T07:09:02.273996+0000","last_fullsized":"2024-09-14T07:09:02.273996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:54:57.601863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":69468295,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":4159,"num_read_kb":7826,"num_write":167,"num_write_kb":9312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"76'104","reported_seq":1439,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267622+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:00.267622+0000","last_peered":"2024-09-14T07:09:00.267622+0000","last_clean":"2024-09-14T07:09:00.267622+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:00.267622+0000","last_undegraded":"2024-09-14T07:09:00.267622+0000","last_fullsized":"2024-09-14T07:09:00.267622+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:04:19.309443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":21731419,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":1309,"num_read_kb":8735,"num_write":121,"num_write_kb":2625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"76'127","reported_seq":1572,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271267+0000","last_change":"2024-09-14T07:08:52.234551+0000","last_active":"2024-09-14T07:09:00.271267+0000","last_peered":"2024-09-14T07:09:00.271267+0000","last_clean":"2024-09-14T07:09:00.271267+0000","last_became_active":"2024-09-14T07:08:52.234002+0000","last_became_peered":"2024-09-14T07:08:52.234002+0000","last_unstale":"2024-09-14T07:09:00.271267+0000","last_undegraded":"2024-09-14T07:09:00.271267+0000","last_fullsized":"2024-09-14T07:09:00.271267+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:52:57.941656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":48335920,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1373,"num_read_kb":4835,"num_write":250,"num_write_kb":13197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1118768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"74'62","reported_seq":646,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267592+0000","last_change":"2024-09-14T07:08:49.269903+0000","last_active":"2024-09-14T07:09:00.267592+0000","last_peered":"2024-09-14T07:09:00.267592+0000","last_clean":"2024-09-14T07:09:00.267592+0000","last_became_active":"2024-09-14T07:08:49.218168+0000","last_became_peered":"2024-09-14T07:08:49.218168+0000","last_unstale":"2024-09-14T07:09:00.267592+0000","last_undegraded":"2024-09-14T07:09:00.267592+0000","last_fullsized":"2024-09-14T07:09:00.267592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:53:21.570579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":4509734,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":387,"num_read_kb":4056,"num_write":70,"num_write_kb":2010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":503846,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"76'141","reported_seq":2417,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267579+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:00.267579+0000","last_peered":"2024-09-14T07:09:00.267579+0000","last_clean":"2024-09-14T07:09:00.267579+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:00.267579+0000","last_undegraded":"2024-09-14T07:09:00.267579+0000","last_fullsized":"2024-09-14T07:09:00.267579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:21:34.044177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":33319974,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":10,"num_read":2326,"num_read_kb":20290,"num_write":140,"num_write_kb":3692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"76'126","reported_seq":2791,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271048+0000","last_change":"2024-09-14T07:08:49.770980+0000","last_active":"2024-09-14T07:09:00.271048+0000","last_peered":"2024-09-14T07:09:00.271048+0000","last_clean":"2024-09-14T07:09:00.271048+0000","last_became_active":"2024-09-14T07:08:49.213956+0000","last_became_peered":"2024-09-14T07:08:49.213956+0000","last_unstale":"2024-09-14T07:09:00.271048+0000","last_undegraded":"2024-09-14T07:09:00.271048+0000","last_fullsized":"2024-09-14T07:09:00.271048+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:07:11.974717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":62939185,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":2806,"num_read_kb":34536,"num_write":210,"num_write_kb":12544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":8864817,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"76'146","reported_seq":4527,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271032+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:00.271032+0000","last_peered":"2024-09-14T07:09:00.271032+0000","last_clean":"2024-09-14T07:09:00.271032+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:00.271032+0000","last_undegraded":"2024-09-14T07:09:00.271032+0000","last_fullsized":"2024-09-14T07:09:00.271032+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:49:23.542131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":53464080,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":4649,"num_read_kb":18460,"num_write":117,"num_write_kb":3299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271254+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:00.271254+0000","last_peered":"2024-09-14T07:09:00.271254+0000","last_clean":"2024-09-14T07:09:00.271254+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:00.271254+0000","last_undegraded":"2024-09-14T07:09:00.271254+0000","last_fullsized":"2024-09-14T07:09:00.271254+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"76'264","reported_seq":1435,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267780+0000","last_change":"2024-09-14T07:08:52.239987+0000","last_active":"2024-09-14T07:09:00.267780+0000","last_peered":"2024-09-14T07:09:00.267780+0000","last_clean":"2024-09-14T07:09:00.267780+0000","last_became_active":"2024-09-14T07:08:52.239406+0000","last_became_peered":"2024-09-14T07:08:52.239406+0000","last_unstale":"2024-09-14T07:09:00.267780+0000","last_undegraded":"2024-09-14T07:09:00.267780+0000","last_fullsized":"2024-09-14T07:09:00.267780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:00:33.900361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":397545,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":2668,"num_read_kb":2334,"num_write":290,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":239,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.269139+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:00.269139+0000","last_peered":"2024-09-14T07:09:00.269139+0000","last_clean":"2024-09-14T07:09:00.269139+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:00.269139+0000","last_undegraded":"2024-09-14T07:09:00.269139+0000","last_fullsized":"2024-09-14T07:09:00.269139+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'162","reported_seq":1353,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271687+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:00.271687+0000","last_peered":"2024-09-14T07:09:00.271687+0000","last_clean":"2024-09-14T07:09:00.271687+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:00.271687+0000","last_undegraded":"2024-09-14T07:09:00.271687+0000","last_fullsized":"2024-09-14T07:09:00.271687+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:18:23.853525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":20203550,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1795,"num_read_kb":7488,"num_write":199,"num_write_kb":2043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271205+0000","last_change":"2024-09-14T07:08:49.212698+0000","last_active":"2024-09-14T07:09:00.271205+0000","last_peered":"2024-09-14T07:09:00.271205+0000","last_clean":"2024-09-14T07:09:00.271205+0000","last_became_active":"2024-09-14T07:08:49.211419+0000","last_became_peered":"2024-09-14T07:08:49.211419+0000","last_unstale":"2024-09-14T07:09:00.271205+0000","last_undegraded":"2024-09-14T07:09:00.271205+0000","last_fullsized":"2024-09-14T07:09:00.271205+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:07:35.929274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"76'415","reported_seq":2033,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267706+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:00.267706+0000","last_peered":"2024-09-14T07:09:00.267706+0000","last_clean":"2024-09-14T07:09:00.267706+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:00.267706+0000","last_undegraded":"2024-09-14T07:09:00.267706+0000","last_fullsized":"2024-09-14T07:09:00.267706+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:18:20.981962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":393271,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":3847,"num_read_kb":3595,"num_write":617,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267754+0000","last_change":"2024-09-14T07:08:49.213968+0000","last_active":"2024-09-14T07:09:00.267754+0000","last_peered":"2024-09-14T07:09:00.267754+0000","last_clean":"2024-09-14T07:09:00.267754+0000","last_became_active":"2024-09-14T07:08:49.213320+0000","last_became_peered":"2024-09-14T07:08:49.213320+0000","last_unstale":"2024-09-14T07:09:00.267754+0000","last_undegraded":"2024-09-14T07:09:00.267754+0000","last_fullsized":"2024-09-14T07:09:00.267754+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:35.337724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'45","reported_seq":1780,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267698+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:00.267698+0000","last_peered":"2024-09-14T07:09:00.267698+0000","last_clean":"2024-09-14T07:09:00.267698+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:00.267698+0000","last_undegraded":"2024-09-14T07:09:00.267698+0000","last_fullsized":"2024-09-14T07:09:00.267698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:30.644693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":8269856,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":1520,"num_read_kb":11887,"num_write":55,"num_write_kb":3208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"78'46","reported_seq":733,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.257924+0000","last_change":"2024-09-14T07:08:49.426259+0000","last_active":"2024-09-14T07:09:02.257924+0000","last_peered":"2024-09-14T07:09:02.257924+0000","last_clean":"2024-09-14T07:09:02.257924+0000","last_became_active":"2024-09-14T07:08:49.210033+0000","last_became_peered":"2024-09-14T07:08:49.210033+0000","last_unstale":"2024-09-14T07:09:02.257924+0000","last_undegraded":"2024-09-14T07:09:02.257924+0000","last_fullsized":"2024-09-14T07:09:02.257924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:15:53.121306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":44498944,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":683,"num_read_kb":7221,"num_write":119,"num_write_kb":9844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":19087872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267636+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:00.267636+0000","last_peered":"2024-09-14T07:09:00.267636+0000","last_clean":"2024-09-14T07:09:00.267636+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:00.267636+0000","last_undegraded":"2024-09-14T07:09:00.267636+0000","last_fullsized":"2024-09-14T07:09:00.267636+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"76'143","reported_seq":1724,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267706+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:00.267706+0000","last_peered":"2024-09-14T07:09:00.267706+0000","last_clean":"2024-09-14T07:09:00.267706+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:00.267706+0000","last_undegraded":"2024-09-14T07:09:00.267706+0000","last_fullsized":"2024-09-14T07:09:00.267706+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:24.601129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":3164834,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":2654,"num_read_kb":5086,"num_write":98,"num_write_kb":1367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":113,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271463+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:00.271463+0000","last_peered":"2024-09-14T07:09:00.271463+0000","last_clean":"2024-09-14T07:09:00.271463+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:00.271463+0000","last_undegraded":"2024-09-14T07:09:00.271463+0000","last_fullsized":"2024-09-14T07:09:00.271463+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"78'93","reported_seq":553,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.171547+0000","last_change":"2024-09-14T07:08:49.550839+0000","last_active":"2024-09-14T07:09:02.171547+0000","last_peered":"2024-09-14T07:09:02.171547+0000","last_clean":"2024-09-14T07:09:02.171547+0000","last_became_active":"2024-09-14T07:08:49.220855+0000","last_became_peered":"2024-09-14T07:08:49.220855+0000","last_unstale":"2024-09-14T07:09:02.171547+0000","last_undegraded":"2024-09-14T07:09:02.171547+0000","last_fullsized":"2024-09-14T07:09:02.171547+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:25:29.549402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":13242921,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":531,"num_read_kb":1731,"num_write":94,"num_write_kb":760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":128082,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270475+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:00.270475+0000","last_peered":"2024-09-14T07:09:00.270475+0000","last_clean":"2024-09-14T07:09:00.270475+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:00.270475+0000","last_undegraded":"2024-09-14T07:09:00.270475+0000","last_fullsized":"2024-09-14T07:09:00.270475+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"76'133","reported_seq":1487,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.269176+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:00.269176+0000","last_peered":"2024-09-14T07:09:00.269176+0000","last_clean":"2024-09-14T07:09:00.269176+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:00.269176+0000","last_undegraded":"2024-09-14T07:09:00.269176+0000","last_fullsized":"2024-09-14T07:09:00.269176+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:40.510138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":42720288,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":1254,"num_read_kb":3239,"num_write":115,"num_write_kb":1256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":157,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267488+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:00.267488+0000","last_peered":"2024-09-14T07:09:00.267488+0000","last_clean":"2024-09-14T07:09:00.267488+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:00.267488+0000","last_undegraded":"2024-09-14T07:09:00.267488+0000","last_fullsized":"2024-09-14T07:09:00.267488+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":239,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270579+0000","last_change":"2024-09-14T07:08:49.338418+0000","last_active":"2024-09-14T07:09:00.270579+0000","last_peered":"2024-09-14T07:09:00.270579+0000","last_clean":"2024-09-14T07:09:00.270579+0000","last_became_active":"2024-09-14T07:08:49.217675+0000","last_became_peered":"2024-09-14T07:08:49.217675+0000","last_unstale":"2024-09-14T07:09:00.270579+0000","last_undegraded":"2024-09-14T07:09:00.270579+0000","last_fullsized":"2024-09-14T07:09:00.270579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:11:06.519350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"74'186","reported_seq":6412,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271235+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:00.271235+0000","last_peered":"2024-09-14T07:09:00.271235+0000","last_clean":"2024-09-14T07:09:00.271235+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:00.271235+0000","last_undegraded":"2024-09-14T07:09:00.271235+0000","last_fullsized":"2024-09-14T07:09:00.271235+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:00:50.807270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":113640500,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":6020,"num_read_kb":52248,"num_write":337,"num_write_kb":22649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"78'338","reported_seq":3384,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-14T07:09:02.236785+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:02.236785+0000","last_peered":"2024-09-14T07:09:02.236785+0000","last_clean":"2024-09-14T07:09:02.236785+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:02.236785+0000","last_undegraded":"2024-09-14T07:09:02.236785+0000","last_fullsized":"2024-09-14T07:09:02.236785+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:07:14.427467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":62256396,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":4598,"num_read_kb":22890,"num_write":389,"num_write_kb":5452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"76'70","reported_seq":469,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270943+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:00.270943+0000","last_peered":"2024-09-14T07:09:00.270943+0000","last_clean":"2024-09-14T07:09:00.270943+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:00.270943+0000","last_undegraded":"2024-09-14T07:09:00.270943+0000","last_fullsized":"2024-09-14T07:09:00.270943+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:11:18.610572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":2156560,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":240,"num_read_kb":1740,"num_write":77,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"77'435","reported_seq":4639,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267919+0000","last_change":"2024-09-14T07:08:52.240372+0000","last_active":"2024-09-14T07:09:00.267919+0000","last_peered":"2024-09-14T07:09:00.267919+0000","last_clean":"2024-09-14T07:09:00.267919+0000","last_became_active":"2024-09-14T07:08:52.239477+0000","last_became_peered":"2024-09-14T07:08:52.239477+0000","last_unstale":"2024-09-14T07:09:00.267919+0000","last_undegraded":"2024-09-14T07:09:00.267919+0000","last_fullsized":"2024-09-14T07:09:00.267919+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:35:19.679463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":43412716,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":6289,"num_read_kb":26177,"num_write":611,"num_write_kb":10494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":8521964,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"76'166","reported_seq":2866,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267640+0000","last_change":"2024-09-14T07:08:52.240897+0000","last_active":"2024-09-14T07:09:00.267640+0000","last_peered":"2024-09-14T07:09:00.267640+0000","last_clean":"2024-09-14T07:09:00.267640+0000","last_became_active":"2024-09-14T07:08:52.239622+0000","last_became_peered":"2024-09-14T07:08:52.239622+0000","last_unstale":"2024-09-14T07:09:00.267640+0000","last_undegraded":"2024-09-14T07:09:00.267640+0000","last_fullsized":"2024-09-14T07:09:00.267640+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:23:14.104906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":29643264,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":3810,"num_read_kb":13000,"num_write":118,"num_write_kb":2863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":476160,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"75'105","reported_seq":2371,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.271080+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:00.271080+0000","last_peered":"2024-09-14T07:09:00.271080+0000","last_clean":"2024-09-14T07:09:00.271080+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:00.271080+0000","last_undegraded":"2024-09-14T07:09:00.271080+0000","last_fullsized":"2024-09-14T07:09:00.271080+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:39:24.368465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":21025792,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":2164,"num_read_kb":10714,"num_write":93,"num_write_kb":1970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"76'193","reported_seq":2098,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267832+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:00.267832+0000","last_peered":"2024-09-14T07:09:00.267832+0000","last_clean":"2024-09-14T07:09:00.267832+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:00.267832+0000","last_undegraded":"2024-09-14T07:09:00.267832+0000","last_fullsized":"2024-09-14T07:09:00.267832+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:26.927492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":18718775,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":2847,"num_read_kb":9986,"num_write":147,"num_write_kb":2786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"76'132","reported_seq":927,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267712+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:00.267712+0000","last_peered":"2024-09-14T07:09:00.267712+0000","last_clean":"2024-09-14T07:09:00.267712+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:00.267712+0000","last_undegraded":"2024-09-14T07:09:00.267712+0000","last_fullsized":"2024-09-14T07:09:00.267712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:30:23.788694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":24046172,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":964,"num_read_kb":5000,"num_write":270,"num_write_kb":13588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"75'93","reported_seq":907,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.267481+0000","last_change":"2024-09-14T07:08:49.325783+0000","last_active":"2024-09-14T07:09:00.267481+0000","last_peered":"2024-09-14T07:09:00.267481+0000","last_clean":"2024-09-14T07:09:00.267481+0000","last_became_active":"2024-09-14T07:08:49.218113+0000","last_became_peered":"2024-09-14T07:08:49.218113+0000","last_unstale":"2024-09-14T07:09:00.267481+0000","last_undegraded":"2024-09-14T07:09:00.267481+0000","last_fullsized":"2024-09-14T07:09:00.267481+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:46:25.951257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":16117248,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":4,"num_read":653,"num_read_kb":1355,"num_write":188,"num_write_kb":12452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":41984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"76'210","reported_seq":2733,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270506+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:00.270506+0000","last_peered":"2024-09-14T07:09:00.270506+0000","last_clean":"2024-09-14T07:09:00.270506+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:00.270506+0000","last_undegraded":"2024-09-14T07:09:00.270506+0000","last_fullsized":"2024-09-14T07:09:00.270506+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:30.416532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":18952847,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":3274,"num_read_kb":4789,"num_write":152,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"76'67","reported_seq":1448,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.270506+0000","last_change":"2024-09-14T07:08:49.288841+0000","last_active":"2024-09-14T07:09:00.270506+0000","last_peered":"2024-09-14T07:09:00.270506+0000","last_clean":"2024-09-14T07:09:00.270506+0000","last_became_active":"2024-09-14T07:08:49.217146+0000","last_became_peered":"2024-09-14T07:08:49.217146+0000","last_unstale":"2024-09-14T07:09:00.270506+0000","last_undegraded":"2024-09-14T07:09:00.270506+0000","last_fullsized":"2024-09-14T07:09:00.270506+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:12:16.864389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":34771456,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":1132,"num_read_kb":5990,"num_write":65,"num_write_kb":1079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":736768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"76'118","reported_seq":2021,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-14T07:09:00.268023+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:00.268023+0000","last_peered":"2024-09-14T07:09:00.268023+0000","last_clean":"2024-09-14T07:09:00.268023+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:00.268023+0000","last_undegraded":"2024-09-14T07:09:00.268023+0000","last_fullsized":"2024-09-14T07:09:00.268023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:12:37.481949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":63319568,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":3195,"num_read_kb":34994,"num_write":124,"num_write_kb":3416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1096994612,"num_objects":506,"num_object_clones":76,"num_object_copies":1012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":506,"num_whiteouts":164,"num_read":83727,"num_read_kb":425826,"num_write":6593,"num_write_kb":178420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":630,"num_bytes_recovered":79253055,"num_keys_recovered":209,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22605824,"data_stored":263952664,"data_compressed":7458876,"data_compressed_allocated":18964480,"data_compressed_original":261818368,"omap_allocated":0,"internal_metadata":0},"log_size":5287,"ondisk_log_size":5287,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":19008,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49800,"kb_used_data":6128,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94322040,"statfs":{"total":96636764160,"available":96585768960,"internally_reserved":0,"allocated":6275072,"data_stored":66670968,"data_compressed":1475410,"data_compressed_allocated":4481024,"data_compressed_original":65751552,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607574,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41996,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329844,"statfs":{"total":96636764160,"available":96593760256,"internally_reserved":0,"allocated":2547712,"data_stored":12741793,"data_compressed":668708,"data_compressed_allocated":1294336,"data_compressed_original":12179456,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607573,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45428,"kb_used_data":5924,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326412,"statfs":{"total":96636764160,"available":96590245888,"internally_reserved":0,"allocated":6066176,"data_stored":60124279,"data_compressed":1812355,"data_compressed_allocated":4354048,"data_compressed_original":59248640,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607575,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44764,"kb_used_data":5244,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327076,"statfs":{"total":96636764160,"available":96590925824,"internally_reserved":0,"allocated":5369856,"data_stored":50426604,"data_compressed":1590185,"data_compressed_allocated":3751936,"data_compressed_original":49639936,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607575,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44300,"kb_used_data":4780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327540,"statfs":{"total":96636764160,"available":96591400960,"internally_reserved":0,"allocated":4894720,"data_stored":53956528,"data_compressed":1171306,"data_compressed_allocated":3428352,"data_compressed_original":53307904,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":51539607575,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1437696,"data_stored":17352863,"data_compressed":558105,"data_compressed_allocated":1204224,"data_compressed_original":17242624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3969024,"data_stored":53015618,"data_compressed":1164414,"data_compressed_allocated":3391488,"data_compressed_original":52718080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4489216,"data_stored":50076081,"data_compressed":1590185,"data_compressed_allocated":3751936,"data_compressed_original":49639936,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5128192,"data_stored":59183388,"data_compressed":1806297,"data_compressed_allocated":4317184,"data_compressed_original":58675200,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5918720,"data_stored":71933463,"data_compressed":1671167,"data_compressed_allocated":5005312,"data_compressed_original":71363072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1662976,"data_stored":12391251,"data_compressed":668708,"data_compressed_allocated":1294336,"data_compressed_original":12179456,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:04.030 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T07:09:04.030 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:04.030 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-14T07:09:04.030 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:09:04.031 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:09:04.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-14T07:09:04.080 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:04.081 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:04.090 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:04.084+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:04.095 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:04.193 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:04.188+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:04.195 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T07:09:04.195 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:04.259 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:04.259 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:04.273 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:04.294 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:04.288+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:04.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:09:04.394 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:04.388+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:04.441 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:04.441 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:04.456 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:04.503 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:04.500+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:04.596 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:04.592+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:04.630 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:04.630 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:04.645 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:04.697 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:04.692+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:04.798 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:04.792+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:04.811 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:04.811 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:04.826 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:04.898 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:04.892+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:04.951 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:04.951 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:04.966 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:05.012 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:05.008+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:05.070 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T07:09:05.100 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:05.096+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:05.136 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:05.137 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:05.150 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:05.201 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:05.196+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:05.278 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:05.278 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:05.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:05.318 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:09:05.318 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:05.312+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:05.402 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:05.396+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:05.439 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:05.439 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:05.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:05.543 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:05.540+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:05.604 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:05.600+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:05.621 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:05.621 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:05.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:05.705 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:05.700+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:05.806 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:05.800+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:05.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:05.810 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:05.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:05.907 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:05.904+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:05.966 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:05.966 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:05.983 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:05.983 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:06.001 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:06.004 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:06.004 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:06.004 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:06.004 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:06.008 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:06.004+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:06.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:06.023 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:06.044 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:06.065 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:06.066 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:06.131 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:06.128+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:06.140 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:06.140 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:06.140 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:06.140 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:06.140 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:06.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:06.210 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:06.204+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:06.285 INFO:teuthology.orchestra.run.smithi145.stdout:2594 read 0x563ce00 thru 0x5647501 (0xa702 bytes) 2024-09-14T07:09:06.296 INFO:teuthology.orchestra.run.smithi145.stdout:2595 read 0x7099200 thru 0x70a1095 (0x7e96 bytes) 2024-09-14T07:09:06.305 INFO:teuthology.orchestra.run.smithi145.stdout:2598 read 0x43e6600 thru 0x43f0af4 (0xa4f5 bytes) 2024-09-14T07:09:06.310 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:06.304+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:06.312 INFO:teuthology.orchestra.run.smithi145.stdout:2599 write 0x4b7c000 thru 0x4b81e04 (0x5e05 bytes) 2024-09-14T07:09:06.314 INFO:teuthology.orchestra.run.smithi145.stdout:2601 write 0x780e00 thru 0x78f7e5 (0xe9e6 bytes) 2024-09-14T07:09:06.316 INFO:teuthology.orchestra.run.smithi145.stdout:2603 punch from 0x21f0e00 to 0x21fb000, (0xa200 bytes) 2024-09-14T07:09:06.324 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:06.324 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:06.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:06.324 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1904 3.b 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 9175040~46080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 49.034837734, 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.101088252, 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1904 2024-09-14T07:09:06.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288893+0000", 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288894+0000", 2024-09-14T07:09:06.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.666e-06 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315149+0000", 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026254493 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315177+0000", 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315200+0000", 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2949e-05 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315307+0000", 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107222 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.315483+0000", 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017589 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:06.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.317437+0000", 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019539890000000002 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.384167+0000", 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066730085999999994 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389856+0000", 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056887079999999998 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389925+0000", 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8977e-05 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389981+0000", 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6158e-05 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.330 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 44.643945365999997, 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:06.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:06.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4071 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18219008~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892767+0000", 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.430963688, 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080854407000000003, 2024-09-14T07:09:06.335 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4071 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892767+0000", 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892767+0000", 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892768+0000", 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.032e-06 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892775+0000", 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9330000000000004e-06 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892795+0000", 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0338000000000001e-05 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892808+0000", 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2717e-05 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973302+0000", 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080493947999999996 2024-09-14T07:09:06.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973456+0000", 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015434199999999999 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973621+0000", 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016509700000000001 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4072 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18513920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892826+0000", 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.430904151, 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081162628000000001, 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4072 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892826+0000", 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892826+0000", 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892827+0000", 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0750000000000001e-06 2024-09-14T07:09:06.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892832+0000", 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7539999999999997e-06 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892850+0000", 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8077e-05 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892864+0000", 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4385e-05 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973672+0000", 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080807244 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973812+0000", 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014033700000000001 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.973989+0000", 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176756 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.343 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4073 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18808832~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892901+0000", 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.430829228, 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081457293, 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4073 2024-09-14T07:09:06.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892901+0000", 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892901+0000", 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892902+0000", 2024-09-14T07:09:06.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.094e-06 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892910+0000", 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9899999999999997e-06 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892934+0000", 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3867999999999999e-05 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892947+0000", 2024-09-14T07:09:06.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3096e-05 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974040+0000", 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081092483000000007 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974180+0000", 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014082999999999999 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974358+0000", 2024-09-14T07:09:06.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000177932 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4074 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18874368~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.892974+0000", 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.43075644, 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081755193000000004, 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4074 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892974+0000", 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892974+0000", 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892975+0000", 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892981+0000", 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9349999999999999e-06 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.892997+0000", 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5996999999999999e-05 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893026+0000", 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9169000000000001e-05 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974408+0000", 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081382198000000003 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974547+0000", 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013913 2024-09-14T07:09:06.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974729+0000", 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018163899999999999 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4075 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19103744~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.893057+0000", 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.430673012, 2024-09-14T07:09:06.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082041696999999997, 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4075 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893057+0000", 2024-09-14T07:09:06.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893057+0000", 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893058+0000", 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0100000000000001e-06 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893064+0000", 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893082+0000", 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8827000000000001e-05 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893097+0000", 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.466e-05 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974778+0000", 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081680536999999998 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.974915+0000", 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013752099999999999 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975099+0000", 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018398199999999999 2024-09-14T07:09:06.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4076 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19169280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.893124+0000", 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.430606659, 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082343668999999994, 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4076 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893124+0000", 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893124+0000", 2024-09-14T07:09:06.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893125+0000", 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.95e-07 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893131+0000", 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2230000000000004e-06 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893149+0000", 2024-09-14T07:09:06.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7694e-05 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893163+0000", 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.468e-05 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975149+0000", 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081985303999999995 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975288+0000", 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013894999999999999 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975467+0000", 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017982299999999999 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4077 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19464192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.893182+0000", 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.430548097000001, 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082699184999999995, 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4077 2024-09-14T07:09:06.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893182+0000", 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893182+0000", 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893183+0000", 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1650000000000001e-06 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893188+0000", 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.549e-06 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893204+0000", 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5534999999999999e-05 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.893220+0000", 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6617e-05 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975569+0000", 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082349011999999999 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975708+0000", 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013850599999999999 2024-09-14T07:09:06.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.975881+0000", 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017380100000000001 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4195 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17285120~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.366 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899732+0000", 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423998486, 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080966627999999999, 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4195 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899732+0000", 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899732+0000", 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899733+0000", 2024-09-14T07:09:06.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6700000000000002e-07 2024-09-14T07:09:06.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899736+0000", 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9059999999999998e-06 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899743+0000", 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5220000000000001e-06 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899752+0000", 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0219999999999993e-06 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980372+0000", 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.080620118000000004 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980531+0000", 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015877299999999999 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980699+0000", 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016741999999999999 2024-09-14T07:09:06.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4196 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17350656~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899761+0000", 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423968871, 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081365215000000005, 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4196 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899761+0000", 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899761+0000", 2024-09-14T07:09:06.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899762+0000", 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7199999999999999e-07 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899765+0000", 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6120000000000001e-06 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899772+0000", 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2849999999999997e-06 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899778+0000", 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980805+0000", 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081027063999999996 2024-09-14T07:09:06.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.980949+0000", 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014400799999999999 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981127+0000", 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017771599999999999 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4197 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17645568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423944798999999, 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081746397999999998, 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4197 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.377 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899786+0000", 2024-09-14T07:09:06.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899789+0000", 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7350000000000001e-06 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899796+0000", 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1180000000000003e-06 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899802+0000", 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9170000000000003e-06 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981211+0000", 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081408736999999995 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981364+0000", 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015300700000000001 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981532+0000", 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016837199999999999 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.381 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4198 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 17940480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899857+0000", 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423873616, 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082043290000000005, 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4198 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899857+0000", 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899857+0000", 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899858+0000", 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0500000000000002e-07 2024-09-14T07:09:06.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899862+0000", 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2490000000000003e-06 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899874+0000", 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1603000000000001e-05 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899881+0000", 2024-09-14T07:09:06.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3300000000000001e-06 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981583+0000", 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081701766999999995 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981716+0000", 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013313100000000001 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.981900+0000", 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000184305 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4199 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18235392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.899975+0000", 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423755084, 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082337240000000006, 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4199 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899975+0000", 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899975+0000", 2024-09-14T07:09:06.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899977+0000", 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.638e-06 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.899984+0000", 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5440000000000001e-06 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900041+0000", 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6910000000000002e-05 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900060+0000", 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8159e-05 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982003+0000", 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.081943960999999996 2024-09-14T07:09:06.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982142+0000", 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013828899999999999 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982313+0000", 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017073899999999999 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4200 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18530304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900204+0000", 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423526404, 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082508573000000002, 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4200 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.391 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900204+0000", 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900204+0000", 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900206+0000", 2024-09-14T07:09:06.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7710000000000001e-06 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900214+0000", 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.456e-06 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900235+0000", 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-05 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900270+0000", 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4581e-05 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982396+0000", 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082126331999999996 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982547+0000", 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000151237 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982713+0000", 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526600000000001 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4201 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18825216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900295+0000", 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423435366, 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082796755, 2024-09-14T07:09:06.395 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4201 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900295+0000", 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900295+0000", 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900296+0000", 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1400000000000001e-06 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900303+0000", 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3770000000000001e-06 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900319+0000", 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6657e-05 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900340+0000", 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1118999999999999e-05 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982803+0000", 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082462368999999994 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.982929+0000", 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126823 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983092+0000", 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016227 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4202 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 18890752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900364+0000", 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423366552999999, 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083126310999999994, 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4202 2024-09-14T07:09:06.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900364+0000", 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900364+0000", 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900365+0000", 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.04e-06 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900370+0000", 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.045e-06 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900406+0000", 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5914000000000002e-05 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900420+0000", 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4012999999999999e-05 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983174+0000", 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.082754105999999994 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983326+0000", 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015226000000000001 2024-09-14T07:09:06.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983490+0000", 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000163933 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4203 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19120128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900469+0000", 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.42326138, 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083390407, 2024-09-14T07:09:06.404 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4203 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900469+0000", 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900469+0000", 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900471+0000", 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.832e-06 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900477+0000", 2024-09-14T07:09:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900498+0000", 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0307000000000001e-05 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900511+0000", 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3298999999999999e-05 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983579+0000", 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083067618999999995 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983704+0000", 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012488899999999999 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983859+0000", 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015579299999999999 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4204 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19185664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900544+0000", 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423186659000001, 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083738039, 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4204 2024-09-14T07:09:06.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900544+0000", 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900544+0000", 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900545+0000", 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0640000000000001e-06 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900552+0000", 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9029999999999998e-06 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900574+0000", 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1974e-05 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900587+0000", 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3492999999999999e-05 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.983941+0000", 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083354182999999998 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984095+0000", 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015380600000000001 2024-09-14T07:09:06.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984282+0000", 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018661599999999999 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4916.0:4205 3.13 3:c9dae399:::rbd_data.11613d73da60.0000000000000000:10 [read 19480576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:53.900631+0000", 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.423099363, 2024-09-14T07:09:06.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.084031554999999994, 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4916", 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1549722642", 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4205 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900631+0000", 2024-09-14T07:09:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900631+0000", 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900632+0000", 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4900000000000005e-07 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900636+0000", 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1030000000000004e-06 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900653+0000", 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6671999999999999e-05 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.900660+0000", 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3819999999999998e-06 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984370+0000", 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.083710190000000004 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984497+0000", 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012703399999999999 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:53.984663+0000", 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016532499999999999 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.417 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.418 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.418 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:06.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:06.420 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:06.412+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:06.520 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:06.516+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:06.602 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:06.602 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:06.602 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:06.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:06.621 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:06.616+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:06.722 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:06.716+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:06.756 INFO:teuthology.orchestra.run.smithi145.stdout:2604 trunc from 0x9a8e600 to 0x892fa00 2024-09-14T07:09:06.797 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:06.797 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:06.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:06.855 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:06.852+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:06.924 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:06.920+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:06.951 INFO:teuthology.orchestra.run.smithi145.stdout:2605 punch from 0x58ccc00 to 0x58d2000, (0x5400 bytes) 2024-09-14T07:09:06.989 INFO:teuthology.orchestra.run.smithi145.stdout:2607 read 0x6d05400 thru 0x6d0e5ea (0x91eb bytes) 2024-09-14T07:09:06.990 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:06.990 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:06.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:06.990 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:06.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.990 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:06.990 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 49.688485002, 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:09:06.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:09:06.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:09:06.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 49.643494695999998, 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:09:06.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:09:06.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.341562240000002, 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:07.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.020234506000001, 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.007 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:09:07.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:09:07.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:09:07.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:09:07.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.58310610699999998, 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:07.016 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.58309544899999999, 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.58308043499999995, 2024-09-14T07:09:07.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.58306952000000001, 2024-09-14T07:09:07.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:09:07.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:09:07.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.58305801000000002, 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:09:07.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.58304546700000004, 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:09:07.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57620946500000003, 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.576198871, 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:09:07.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.045 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57615108999999998, 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:09:07.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:09:07.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.049 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.051 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.051 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57613762300000004, 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:07.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:07.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:09:07.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:09:07.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57612665100000005, 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.060 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57611588899999999, 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57609983499999995, 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:09:07.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57608326399999998, 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.073 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57607148799999996, 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.077 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57606081099999995, 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.078 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:09:07.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:09:07.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:09:07.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:09:07.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.082 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:07.082 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:07.082 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:07.084 INFO:teuthology.orchestra.run.smithi145.stdout:2608 read 0x4668e00 thru 0x467742e (0xe62f bytes) 2024-09-14T07:09:07.084 INFO:teuthology.orchestra.run.smithi145.stdout:2610 write 0x70d1c00 thru 0x70dfd79 (0xe17a bytes) 2024-09-14T07:09:07.084 INFO:teuthology.orchestra.run.smithi145.stdout:2612 write 0x7600 thru 0x13ec1 (0xc8c2 bytes) 2024-09-14T07:09:07.084 INFO:teuthology.orchestra.run.smithi145.stdout:2613 write 0x7637a00 thru 0x763ba41 (0x4042 bytes) 2024-09-14T07:09:07.084 INFO:teuthology.orchestra.run.smithi145.stdout:2614 punch from 0x4713200 to 0x4715800, (0x2600 bytes) 2024-09-14T07:09:07.084 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:07.085 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:07.080+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:07.184 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:07.180+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:07.280 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12327 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.208834+0000", 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.067278698999999997, 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067299226000000004, 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.281 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12327 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208834+0000", 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208834+0000", 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208834+0000", 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0100000000000004e-07 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208843+0000", 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2719999999999997e-06 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208850+0000", 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:09:07.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208855+0000", 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4040000000000003e-06 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12461 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217315+0000", 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058797215, 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.058887443999999997, 2024-09-14T07:09:07.284 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12461 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217315+0000", 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217315+0000", 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217316+0000", 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217318+0000", 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1490000000000001e-06 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217324+0000", 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217330+0000", 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7930000000000003e-06 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12328 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.208880+0000", 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.067231867000000001, 2024-09-14T07:09:07.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067381348999999993, 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12328 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208880+0000", 2024-09-14T07:09:07.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208880+0000", 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208881+0000", 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3600000000000003e-07 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208884+0000", 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.525e-06 2024-09-14T07:09:07.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208915+0000", 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1199999999999999e-05 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208922+0000", 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0219999999999999e-06 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.290 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12462 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217336+0000", 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058776109999999999, 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.058978236000000003, 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12462 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217336+0000", 2024-09-14T07:09:07.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217336+0000", 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217337+0000", 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8200000000000001e-07 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217339+0000", 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.956e-06 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217345+0000", 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1140000000000002e-06 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217350+0000", 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9490000000000004e-06 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12329 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.208958+0000", 2024-09-14T07:09:07.293 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.067154537, 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067407333, 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12329 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208958+0000", 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208958+0000", 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208959+0000", 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5600000000000005e-07 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208962+0000", 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2270000000000002e-06 2024-09-14T07:09:07.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208970+0000", 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3599999999999996e-06 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208977+0000", 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3419999999999996e-06 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12463 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.296 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217356+0000", 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058756452000000001, 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.059061899000000001, 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12463 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217356+0000", 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217356+0000", 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217356+0000", 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217358+0000", 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9190000000000002e-06 2024-09-14T07:09:07.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217365+0000", 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1269999999999997e-06 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217370+0000", 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.78e-06 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12330 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.208987+0000", 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.067125116999999998, 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067482576000000002, 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12330 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.300 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208987+0000", 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208987+0000", 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208988+0000", 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3900000000000005e-07 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208990+0000", 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2330000000000001e-06 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.208997+0000", 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209002+0000", 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3159999999999996e-06 2024-09-14T07:09:07.302 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12331 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.209010+0000", 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.067102269000000006, 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067511290000000002, 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12331 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209010+0000", 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209010+0000", 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209011+0000", 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6499999999999999e-07 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209013+0000", 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0729999999999999e-06 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209019+0000", 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2890000000000003e-06 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209026+0000", 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.058e-06 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12332 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.209034+0000", 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.067078604, 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067537983999999995, 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12332 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209034+0000", 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209034+0000", 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209034+0000", 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6000000000000004e-07 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209036+0000", 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1629999999999999e-06 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209043+0000", 2024-09-14T07:09:07.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6510000000000004e-06 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.209048+0000", 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3240000000000002e-06 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12333 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.214377+0000", 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.061735448999999998, 2024-09-14T07:09:07.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.062248271000000001, 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12333 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214377+0000", 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214377+0000", 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214378+0000", 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.325e-06 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214383+0000", 2024-09-14T07:09:07.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0749999999999997e-06 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214396+0000", 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2745000000000001e-05 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214425+0000", 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9303999999999999e-05 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.312 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12334 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.214453+0000", 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.061659737999999999, 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.062223325000000003, 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12334 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214453+0000", 2024-09-14T07:09:07.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214453+0000", 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214454+0000", 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3500000000000005e-07 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214457+0000", 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8890000000000004e-06 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214484+0000", 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6466999999999999e-05 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.214493+0000", 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9449999999999999e-06 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12840 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.236383+0000", 2024-09-14T07:09:07.315 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.039729335999999997, 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.040347058999999998, 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12840 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.236383+0000", 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.236383+0000", 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237547+0000", 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011640979999999999 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237551+0000", 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1230000000000002e-06 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237561+0000", 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9629999999999993e-06 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237567+0000", 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6540000000000004e-06 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12841 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.318 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237579+0000", 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038533572000000002, 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039201871999999999, 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12841 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237579+0000", 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237579+0000", 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237579+0000", 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237581+0000", 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.629e-06 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237587+0000", 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5840000000000002e-06 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237590+0000", 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.286e-06 2024-09-14T07:09:07.321 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12842 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237595+0000", 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038517521999999998, 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039235622999999997, 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.322 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12842 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237595+0000", 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237595+0000", 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237595+0000", 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6600000000000003e-07 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237596+0000", 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.358e-06 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237600+0000", 2024-09-14T07:09:07.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7309999999999999e-06 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237603+0000", 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9670000000000001e-06 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12843 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237635+0000", 2024-09-14T07:09:07.325 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038477457999999999, 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039246343000000003, 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12843 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237635+0000", 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237635+0000", 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237635+0000", 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9900000000000001e-07 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237638+0000", 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6510000000000001e-06 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237647+0000", 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5140000000000001e-06 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237806+0000", 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000159724 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12260 3.11 3:89ee3c99:::rbd_data.116150b87d44.0000000000000008:14 [read 10397696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.328 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.207647+0000", 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.068464926999999995, 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.069288229000000007, 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12260 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207647+0000", 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207647+0000", 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207648+0000", 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1799999999999995e-07 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207650+0000", 2024-09-14T07:09:07.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9829999999999999e-06 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207656+0000", 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2219999999999999e-06 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207661+0000", 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9749999999999998e-06 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.276833+0000", 2024-09-14T07:09:07.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.069172177000000001 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.276909+0000", 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5358000000000006e-05 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12844 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237825+0000", 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038287058999999998, 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039169818000000002, 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.332 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12844 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237825+0000", 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237825+0000", 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237826+0000", 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237834+0000", 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7689999999999996e-06 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237847+0000", 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3738e-05 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237854+0000", 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5230000000000001e-06 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12261 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.207668+0000", 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.068443983999999999, 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.069387542999999996, 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.335 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12261 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207668+0000", 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207668+0000", 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207669+0000", 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207671+0000", 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0870000000000001e-06 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207679+0000", 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7840000000000004e-06 2024-09-14T07:09:07.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207684+0000", 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8119999999999998e-06 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12710 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.231337+0000", 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.044775017, 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045770776999999999, 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12710 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231337+0000", 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231337+0000", 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231338+0000", 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231339+0000", 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2249999999999999e-06 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231342+0000", 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.444e-06 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231345+0000", 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8229999999999999e-06 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12845 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237860+0000", 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038252324999999997, 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039298091, 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12845 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237860+0000", 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237860+0000", 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237860+0000", 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4499999999999998e-07 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237862+0000", 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.539e-06 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237867+0000", 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2129999999999999e-06 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237870+0000", 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1049999999999999e-06 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12262 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.207690+0000", 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.068422624000000001, 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.069519112999999993, 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.344 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12262 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207690+0000", 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207690+0000", 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207690+0000", 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207692+0000", 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9130000000000001e-06 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207699+0000", 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9500000000000004e-06 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207704+0000", 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5800000000000002e-06 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12711 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.231349+0000", 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.044763329999999997, 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045910107999999998, 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.347 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12711 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231349+0000", 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231349+0000", 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231349+0000", 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6600000000000003e-07 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231351+0000", 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500000000000001e-06 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231354+0000", 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4599999999999999e-06 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231357+0000", 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12846 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237874+0000", 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038237963, 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039436084000000003, 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12846 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237874+0000", 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237874+0000", 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237875+0000", 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9900000000000002e-07 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237876+0000", 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237881+0000", 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8330000000000001e-06 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237884+0000", 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0429999999999999e-06 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12263 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.207711+0000", 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.068401634000000003, 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.069650789000000005, 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12263 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207711+0000", 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207711+0000", 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207711+0000", 2024-09-14T07:09:07.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0399999999999996e-07 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207713+0000", 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1040000000000001e-06 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207720+0000", 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2859999999999996e-06 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.207725+0000", 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.152e-06 2024-09-14T07:09:07.355 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12712 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.231360+0000", 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.044752023000000002, 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.046052456999999998, 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12712 2024-09-14T07:09:07.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231360+0000", 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231360+0000", 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231361+0000", 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6800000000000001e-07 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231362+0000", 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1850000000000001e-06 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231365+0000", 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4510000000000001e-06 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231368+0000", 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.358 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12847 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237888+0000", 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038224582, 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039575626000000003, 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.359 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12847 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237888+0000", 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237888+0000", 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237888+0000", 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237889+0000", 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3239999999999999e-06 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237894+0000", 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5549999999999996e-06 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237897+0000", 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.864e-06 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12713 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.231915+0000", 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.044197680000000003, 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045599819, 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12713 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231915+0000", 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231915+0000", 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231915+0000", 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2500000000000005e-07 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231933+0000", 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7674000000000001e-05 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.231957+0000", 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4131999999999999e-05 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232069+0000", 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011186 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12848 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237917+0000", 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038195071999999997, 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039647874999999999, 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.365 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12848 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237917+0000", 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237917+0000", 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237918+0000", 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237919+0000", 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.556e-06 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237924+0000", 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2170000000000002e-06 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237928+0000", 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2339999999999999e-06 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12714 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.232084+0000", 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.044028633999999997, 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045532248999999997, 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12714 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232084+0000", 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232084+0000", 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232084+0000", 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.06e-07 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232088+0000", 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232096+0000", 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4789999999999996e-06 2024-09-14T07:09:07.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232103+0000", 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1360000000000002e-06 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12582 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.222055+0000", 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.054057235000000002, 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.055611948000000001, 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.371 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12582 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222055+0000", 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222055+0000", 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222056+0000", 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1399999999999996e-07 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222058+0000", 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.306e-06 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222065+0000", 2024-09-14T07:09:07.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8009999999999998e-06 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222070+0000", 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8409999999999999e-06 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12715 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.232114+0000", 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.043998838999999998, 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045605584999999997, 2024-09-14T07:09:07.374 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12715 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232114+0000", 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232114+0000", 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232114+0000", 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0800000000000004e-07 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232116+0000", 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2850000000000001e-06 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232123+0000", 2024-09-14T07:09:07.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.754e-06 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232128+0000", 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2619999999999998e-06 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12849 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.237932+0000", 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.038180743000000003, 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039837628, 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.377 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12849 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237932+0000", 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.237932+0000", 2024-09-14T07:09:07.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240001+0000", 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020697390000000001 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240006+0000", 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2880000000000003e-06 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240020+0000", 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4493e-05 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240028+0000", 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2390000000000002e-06 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12583 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.222078+0000", 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.054034327, 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.055744171000000002, 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12583 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222078+0000", 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222078+0000", 2024-09-14T07:09:07.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222079+0000", 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3499999999999996e-07 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222081+0000", 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1780000000000002e-06 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222088+0000", 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0319999999999998e-06 2024-09-14T07:09:07.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222093+0000", 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.339e-06 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12716 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.232151+0000", 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.043961687999999999, 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045724329000000001, 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12716 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232151+0000", 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232151+0000", 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232151+0000", 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.75e-07 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232154+0000", 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9160000000000001e-06 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232165+0000", 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0811e-05 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232172+0000", 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0319999999999998e-06 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12850 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.240048+0000", 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.036064074000000002, 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037921616999999998, 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12850 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240048+0000", 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240048+0000", 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240049+0000", 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3399999999999999e-07 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240053+0000", 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6160000000000002e-06 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240060+0000", 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3690000000000001e-06 2024-09-14T07:09:07.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240066+0000", 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1800000000000001e-06 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.5104.0:12999 3.3 e82/68 3:c4105e4a:::rbd_data.11615ad0c92d.0000000000000007:head v 82'62, mlcod=82'60)", 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.276980+0000", 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9991326, 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001044531, 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.389 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.276980+0000", 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.276980+0000", 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.276980+0000", 2024-09-14T07:09:07.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0599999999999999e-07 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.277037+0000", 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6557000000000001e-05 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.277049+0000", 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851000000000001e-05 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.277055+0000", 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-14T07:09:07.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.277099+0000", 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3559000000000001e-05 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.277114+0000", 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5054e-05 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.392 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12584 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.222100+0000", 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.054011867999999998, 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.055982932999999999, 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12584 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222100+0000", 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222100+0000", 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222101+0000", 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2600000000000002e-07 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222103+0000", 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.12e-06 2024-09-14T07:09:07.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222110+0000", 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222114+0000", 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8269999999999997e-06 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.395 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12717 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.232182+0000", 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.043930376, 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045953497000000003, 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12717 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232182+0000", 2024-09-14T07:09:07.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232182+0000", 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232183+0000", 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4600000000000005e-07 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232185+0000", 2024-09-14T07:09:07.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6280000000000001e-06 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232194+0000", 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232204+0000", 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2960000000000004e-06 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.398 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12851 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.240075+0000", 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.036037342999999999, 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038113945000000003, 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12851 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240075+0000", 2024-09-14T07:09:07.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240075+0000", 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240076+0000", 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240078+0000", 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.627e-06 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240087+0000", 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5480000000000002e-06 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.240121+0000", 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3877000000000001e-05 2024-09-14T07:09:07.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12585 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.222122+0000", 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.053990354999999997, 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.056118473000000002, 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12585 2024-09-14T07:09:07.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222122+0000", 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222122+0000", 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222122+0000", 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7599999999999997e-07 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.222146+0000", 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.351e-05 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228778+0000", 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066323069999999996 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228793+0000", 2024-09-14T07:09:07.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5088e-05 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12718 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.232801+0000", 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.043311682999999997, 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045493461999999998, 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12718 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232801+0000", 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232801+0000", 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232801+0000", 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9800000000000003e-07 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232805+0000", 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.377e-06 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232814+0000", 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3540000000000006e-06 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232819+0000", 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3349999999999997e-06 2024-09-14T07:09:07.407 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12586 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228810+0000", 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047302084000000001, 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049534090000000003, 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.408 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12586 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228810+0000", 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228810+0000", 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228811+0000", 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000003e-07 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228814+0000", 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228823+0000", 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7220000000000005e-06 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228827+0000", 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0060000000000003e-06 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12719 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.232831+0000", 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.043281527, 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.045565145000000001, 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12719 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232831+0000", 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232831+0000", 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232831+0000", 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8100000000000003e-07 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232834+0000", 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7039999999999999e-06 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232842+0000", 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9850000000000006e-06 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.232848+0000", 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12452 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.216968+0000", 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.05914407, 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.061483180999999998, 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.414 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12452 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216968+0000", 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216968+0000", 2024-09-14T07:09:07.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216969+0000", 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216971+0000", 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1129999999999999e-06 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216978+0000", 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6710000000000002e-06 2024-09-14T07:09:07.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216983+0000", 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9989999999999999e-06 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12587 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228833+0000", 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047279596, 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049669475999999997, 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.417 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12587 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228833+0000", 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228833+0000", 2024-09-14T07:09:07.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228833+0000", 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5499999999999999e-07 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228835+0000", 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7719999999999999e-06 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228839+0000", 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1819999999999999e-06 2024-09-14T07:09:07.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228842+0000", 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0249999999999998e-06 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12720 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.233981+0000", 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.042130860999999999, 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044573768, 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.420 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12720 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.233981+0000", 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.233981+0000", 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.233982+0000", 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1299999999999999e-07 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.233986+0000", 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1949999999999996e-06 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.233999+0000", 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2114e-05 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.234006+0000", 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4540000000000001e-06 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12453 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.216991+0000", 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.059120905000000001, 2024-09-14T07:09:07.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.061615939000000002, 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12453 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216991+0000", 2024-09-14T07:09:07.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216991+0000", 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216992+0000", 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5600000000000001e-07 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.216994+0000", 2024-09-14T07:09:07.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2199999999999999e-06 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217001+0000", 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7839999999999998e-06 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217006+0000", 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4009999999999996e-06 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12588 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228851+0000", 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047261084000000002, 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049811471000000003, 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12588 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228851+0000", 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228851+0000", 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228852+0000", 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.39e-07 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228853+0000", 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228857+0000", 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8959999999999996e-06 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228861+0000", 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4429999999999999e-06 2024-09-14T07:09:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12721 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.234019+0000", 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.042092984, 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.044694721, 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.430 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12721 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.234019+0000", 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.234019+0000", 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.234020+0000", 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9400000000000005e-07 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.234023+0000", 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8930000000000001e-06 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.234031+0000", 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.234037+0000", 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.964e-06 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12454 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217014+0000", 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.059098427000000002, 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.061756204000000002, 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12454 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217014+0000", 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217014+0000", 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217014+0000", 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400000000000001e-07 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217021+0000", 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0330000000000003e-06 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217028+0000", 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0539999999999997e-06 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217034+0000", 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1030000000000001e-06 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12589 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228865+0000", 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047247786999999999, 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.049957185000000001, 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.436 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12589 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228865+0000", 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228865+0000", 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228865+0000", 2024-09-14T07:09:07.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7500000000000001e-07 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228866+0000", 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5260000000000001e-06 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228870+0000", 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6200000000000001e-06 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228873+0000", 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7640000000000001e-06 2024-09-14T07:09:07.438 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12455 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217142+0000", 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058970407000000002, 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.061733731, 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12455 2024-09-14T07:09:07.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217142+0000", 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217142+0000", 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217143+0000", 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5700000000000002e-07 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217146+0000", 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3239999999999999e-06 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217169+0000", 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2971000000000001e-05 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217177+0000", 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7689999999999996e-06 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12590 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228876+0000", 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047235892000000002, 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050051788999999999, 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12590 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228876+0000", 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228876+0000", 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228877+0000", 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000002e-07 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228878+0000", 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3960000000000001e-06 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228883+0000", 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7049999999999998e-06 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228886+0000", 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0469999999999998e-06 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12456 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217188+0000", 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058924329999999997, 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.061794612999999998, 2024-09-14T07:09:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12456 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217188+0000", 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217188+0000", 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217189+0000", 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7999999999999995e-07 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217192+0000", 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.92e-06 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217200+0000", 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8249999999999994e-06 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217206+0000", 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12591 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228890+0000", 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047222743999999997, 2024-09-14T07:09:07.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050143597999999998, 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12591 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228890+0000", 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228890+0000", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228890+0000", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7399999999999999e-07 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228891+0000", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.407e-06 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228895+0000", 2024-09-14T07:09:07.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6849999999999999e-06 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228898+0000", 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.841e-06 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12457 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217233+0000", 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058879366000000002, 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.061853712999999998, 2024-09-14T07:09:07.451 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12457 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217233+0000", 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217233+0000", 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217234+0000", 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217236+0000", 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.475e-06 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217243+0000", 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3629999999999996e-06 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217249+0000", 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.271e-06 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12592 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228903+0000", 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047209813000000003, 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050235414999999999, 2024-09-14T07:09:07.454 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12592 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228903+0000", 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228903+0000", 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228903+0000", 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228905+0000", 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9489999999999999e-06 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228922+0000", 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7385999999999999e-05 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228929+0000", 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6309999999999997e-06 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12458 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217256+0000", 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058855960999999998, 2024-09-14T07:09:07.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.061934922000000003, 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12458 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217256+0000", 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217256+0000", 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217257+0000", 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2e-07 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217259+0000", 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1119999999999999e-06 2024-09-14T07:09:07.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217265+0000", 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2929999999999997e-06 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217270+0000", 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9740000000000001e-06 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12593 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.228947+0000", 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.047165114000000001, 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.050295606, 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12593 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228947+0000", 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228947+0000", 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228948+0000", 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3e-07 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228951+0000", 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0130000000000002e-06 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228957+0000", 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1909999999999997e-06 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.228962+0000", 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12459 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217277+0000", 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058835825000000001, 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.062016544, 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12459 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217277+0000", 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217277+0000", 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217277+0000", 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9799999999999999e-07 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217279+0000", 2024-09-14T07:09:07.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9939999999999999e-06 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217285+0000", 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1120000000000001e-06 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217290+0000", 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.826e-06 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.466 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:12460 3.11 3.993c7791 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.217296+0000", 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.058816225999999999, 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.062051697000000003, 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12460 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217296+0000", 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217296+0000", 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217297+0000", 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217298+0000", 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9290000000000001e-06 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217304+0000", 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9750000000000004e-06 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.217309+0000", 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 61 2024-09-14T07:09:07.469 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:07.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:09:07.471 INFO:teuthology.orchestra.run.smithi145.stdout:2617 trunc from 0x892fa00 to 0x2029400 2024-09-14T07:09:07.472 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:07.472 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:07.464+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:07.494 INFO:teuthology.orchestra.run.smithi145.stdout:2619 clone 19 order 25 su 131072 sc 13 2024-09-14T07:09:07.572 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:07.568+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:07.632 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:07.632 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:07.632 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:07.633 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:07.633 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:07.645 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:07.691 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:07.688+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:07.767 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 95.464464986999999, 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:07.768 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:07.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:07.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.473681851999999, 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:09:07.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:07.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.780 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4703.0:530 3.f e46/21 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head v 46'75, mlcod=45'73)", 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.902582088000003, 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.075870370000000006, 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862389+0000", 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6579999999999998e-06 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870883+0000", 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084939100000000003 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870916+0000", 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2428000000000002e-05 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870931+0000", 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5656000000000001e-05 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871022+0000", 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1063000000000005e-05 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871047+0000", 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4125e-05 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:07.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938212+0000", 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067164920000000003 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938257+0000", 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5609999999999999e-05 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.570471803, 2024-09-14T07:09:07.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:07.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:07.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:07.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:07.793 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.570351924000001, 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:07.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:07.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:07.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.553126804000001, 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.801 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:07.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:07.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:07.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:07.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10172 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112247+0000", 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65272185199999999, 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072580966999999996, 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10172 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112247+0000", 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112247+0000", 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112248+0000", 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2900000000000004e-07 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112257+0000", 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0399999999999998e-06 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112263+0000", 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8979999999999999e-06 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112268+0000", 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7269999999999998e-06 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184668+0000", 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072399986999999999 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184737+0000", 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8813999999999995e-05 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184828+0000", 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0971999999999996e-05 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10174 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 522240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65269209399999994, 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072728104000000002, 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.810 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10174 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7100000000000002e-07 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112280+0000", 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4310000000000001e-06 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112286+0000", 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4540000000000004e-06 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112291+0000", 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.989e-06 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184857+0000", 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072565397000000004 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184922+0000", 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5866999999999998e-05 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185005+0000", 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2395e-05 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10175 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 544768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112299+0000", 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65266954899999996, 2024-09-14T07:09:07.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072880912000000006, 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10175 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112299+0000", 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112299+0000", 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112300+0000", 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.63e-07 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112302+0000", 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.345e-06 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112309+0000", 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112314+0000", 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185033+0000", 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072719324000000002 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185102+0000", 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8714999999999995e-05 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185180+0000", 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8251999999999995e-05 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10177 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 567296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.818 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65264726900000003, 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073321711999999997, 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10177 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7199999999999999e-07 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112324+0000", 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.277e-06 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112331+0000", 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3260000000000001e-06 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112336+0000", 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.916e-06 2024-09-14T07:09:07.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185383+0000", 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073047415000000004 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185490+0000", 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107361 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185643+0000", 2024-09-14T07:09:07.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015294500000000001 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10179 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 589824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112367+0000", 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65260142399999999, 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073522753999999996, 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10179 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112367+0000", 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112367+0000", 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112368+0000", 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4799999999999998e-07 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112371+0000", 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.666e-06 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112387+0000", 2024-09-14T07:09:07.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6413999999999999e-05 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112393+0000", 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5029999999999997e-06 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185694+0000", 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073301253999999996 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185774+0000", 2024-09-14T07:09:07.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0319000000000001e-05 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185890+0000", 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011605 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10180 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 612352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112402+0000", 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65256715799999998, 2024-09-14T07:09:07.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073678974999999994, 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10180 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112402+0000", 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112402+0000", 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112471+0000", 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9459e-05 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112477+0000", 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112486+0000", 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9840000000000007e-06 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112492+0000", 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1179999999999997e-06 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185911+0000", 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073418330000000004 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185974+0000", 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3585999999999997e-05 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186081+0000", 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010649099999999999 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10182 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 634880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112503+0000", 2024-09-14T07:09:07.831 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65246587199999995, 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073876662999999995, 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10182 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112503+0000", 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112503+0000", 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112842+0000", 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033877899999999999 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112847+0000", 2024-09-14T07:09:07.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.904e-06 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112858+0000", 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1610000000000001e-05 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112866+0000", 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8949999999999998e-06 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186123+0000", 2024-09-14T07:09:07.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073257231000000006 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186253+0000", 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012930799999999999 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186380+0000", 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012693599999999999 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.835 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10183 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 657408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112882+0000", 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65208646400000003, 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073654547000000001, 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10183 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112882+0000", 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112882+0000", 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112883+0000", 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5400000000000001e-07 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112886+0000", 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9160000000000001e-06 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112894+0000", 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7689999999999996e-06 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112900+0000", 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1630000000000001e-06 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186399+0000", 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073499373000000007 2024-09-14T07:09:07.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186463+0000", 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3675000000000003e-05 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186537+0000", 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3997000000000003e-05 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10185 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112909+0000", 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65205944999999998, 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073873810999999998, 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10185 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112909+0000", 2024-09-14T07:09:07.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112909+0000", 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113242+0000", 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033220400000000001 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113246+0000", 2024-09-14T07:09:07.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7740000000000004e-06 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113257+0000", 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0749e-05 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113265+0000", 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4340000000000003e-06 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186582+0000", 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073317462999999999 2024-09-14T07:09:07.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186660+0000", 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8092999999999999e-05 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186783+0000", 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123094 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10187 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.113278+0000", 2024-09-14T07:09:07.843 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.651690832, 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073745920000000006, 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10187 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113278+0000", 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113278+0000", 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113279+0000", 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3399999999999998e-07 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113282+0000", 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113289+0000", 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4909999999999999e-06 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113295+0000", 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9089999999999997e-06 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186834+0000", 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073539123999999997 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186913+0000", 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9289000000000006e-05 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187024+0000", 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110444 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10189 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 724992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.847 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.113305+0000", 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65166422800000001, 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073949126000000004, 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10189 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113305+0000", 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113305+0000", 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114430+0000", 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011255099999999999 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114435+0000", 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114450+0000", 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5452e-05 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114459+0000", 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1999999999999994e-06 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187068+0000", 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072609952000000005 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187149+0000", 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0675999999999998e-05 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187254+0000", 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010461199999999999 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10191 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.851 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.114473+0000", 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65049560399999995, 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073037823000000002, 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10191 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114473+0000", 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114473+0000", 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114699+0000", 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022595199999999999 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114704+0000", 2024-09-14T07:09:07.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114713+0000", 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2660000000000007e-06 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114721+0000", 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0949999999999998e-06 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187303+0000", 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072582038000000001 2024-09-14T07:09:07.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187392+0000", 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9924000000000002e-05 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187511+0000", 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011858699999999999 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.855 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10193 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 770048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.114734+0000", 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.65023503100000002, 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073012056000000006, 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10193 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114734+0000", 2024-09-14T07:09:07.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114734+0000", 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115076+0000", 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034258599999999998 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115099+0000", 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2130999999999999e-05 2024-09-14T07:09:07.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115109+0000", 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0008e-05 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115116+0000", 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6650000000000003e-06 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187548+0000", 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072432283 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187632+0000", 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3380999999999993e-05 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187746+0000", 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114002 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.859 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10195 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.115130+0000", 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.64983855599999996, 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072812720999999997, 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10195 2024-09-14T07:09:07.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115130+0000", 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115130+0000", 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115699+0000", 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056919800000000003 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115705+0000", 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115714+0000", 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6339999999999996e-06 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115721+0000", 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.17e-06 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187780+0000", 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072058104999999997 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187848+0000", 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8683999999999995e-05 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:07.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187943+0000", 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4757000000000001e-05 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:07.864 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:07.865 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:07.866 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:07.860+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:07.966 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:07.960+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:08.036 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:08.036 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:08.036 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:08.036 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:08.050 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:08.066 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:08.060+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:08.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:08.167 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:08.164+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:08.219 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:08.219 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:08.219 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:08.219 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:08.219 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:08.233 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:08.242 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:08.243 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:08.259 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:08.259 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:08.268 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:08.264+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:08.275 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:08.275 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:08.294 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:08.312 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:08.312 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:09:08.333 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:08.395 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:08.392+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:08.407 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:08.407 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:08.421 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:08.451 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone18 from 0x2029400 (overlap 0x2029400) to 0x1cf1e00 2024-09-14T07:09:08.470 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:08.464+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:08.568 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:08.568 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:08.570 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:08.564+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:08.585 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:08.671 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:08.664+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:08.711 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:08.711 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:08.725 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:08.772 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:08.768+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:08.873 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:08.868+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:08.887 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:08.887 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:08.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:08.974 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:08.968+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:09.045 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:09.045 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:09.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:09.098 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T07:09:09.107 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:09.104+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:09.176 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:09.172+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:09.210 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:09:09.211 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:09.211 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-14T07:09:09.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-14T07:09:09.229 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:09.229 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:09.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:09.277 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:09.272+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:09.378 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:09.372+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:09.425 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:09.426 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:09.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:09.479 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:09.476+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:09.538 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:09:09.532+0000 7f1e6b537640 -1 mon.a@0(leader).osd e83 definitely_dead 0 2024-09-14T07:09:09.580 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:09.576+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:09.615 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:09.615 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:09.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:09.681 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:09.676+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:09.756 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:09.756 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:09.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:09.781 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:09.776+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:09.882 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:09.876+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:09.949 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:09.949 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:09.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:10.011 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:10.008+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:10.083 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:10.080+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:10.098 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:10.098 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:10.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:10.203 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:10.200+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:10.249 INFO:teuthology.orchestra.run.smithi145.stdout:2623 read 0xaa9000 thru 0xaae9cc (0x59cd bytes) 2024-09-14T07:09:10.277 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:10.277 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:10.285 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:10.280+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:10.289 INFO:teuthology.orchestra.run.smithi145.stdout:2627 trunc from 0x2029400 to 0x887f800 2024-09-14T07:09:10.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.0. 2024-09-14T07:09:10.332 INFO:teuthology.orchestra.run.smithi145.stdout:2630 trunc from 0x887f800 to 0xa04d200 2024-09-14T07:09:10.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:09:10.351 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:10.360 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-14T07:09:10.386 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:10.380+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:10.459 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:10.460 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:10.477 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:10.523 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:10.520+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:10.542 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:10.559 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:10.559 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:10.576 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:10.576 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:10.587 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:10.584+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:10.596 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:10.612 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:10.612 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:10.612 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:10.612 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:10.617 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:10.626 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:10.636 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:10.636 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:10.688 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:10.684+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:10.789 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:10.788+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:10.789 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:10.790 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:10.790 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:10.790 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:10.790 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:10.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:10.890 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:10.888+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 98.626480197999996, 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:10.928 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:10.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:10.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1906 3.15 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 911360~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 53.635697063000002, 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.102810018, 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1906 2024-09-14T07:09:10.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291287+0000", 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.291289+0000", 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.776e-06 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319726+0000", 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0284376 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319769+0000", 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2342999999999998e-05 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319791+0000", 2024-09-14T07:09:10.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1954999999999999e-05 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.319880+0000", 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9722999999999995e-05 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320014+0000", 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133712 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.321857+0000", 2024-09-14T07:09:10.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018424649999999999 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.362916+0000", 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041059768000000003 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.393966+0000", 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031049407000000001 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394036+0000", 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9884999999999999e-05 2024-09-14T07:09:10.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.394097+0000", 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1383999999999994e-05 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4703.0:530 3.f e46/21 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head v 46'75, mlcod=45'73)", 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.064597298999999, 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.075870370000000006, 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862387+0000", 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.862389+0000", 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6579999999999998e-06 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870883+0000", 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084939100000000003 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870916+0000", 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2428000000000002e-05 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.870931+0000", 2024-09-14T07:09:10.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5656000000000001e-05 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871022+0000", 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1063000000000005e-05 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.871047+0000", 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4125e-05 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938212+0000", 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067164920000000003 2024-09-14T07:09:10.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938257+0000", 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5609999999999999e-05 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.732487014, 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:10.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:10.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:10.948 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:10.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.948 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:10.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:10.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:10.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.732367135000001, 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:10.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:10.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:10.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.715142015000001, 2024-09-14T07:09:10.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10172 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 499712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112247+0000", 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8147370629999999, 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072580966999999996, 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10172 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.969 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112247+0000", 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112247+0000", 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112248+0000", 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2900000000000004e-07 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112257+0000", 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0399999999999998e-06 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112263+0000", 2024-09-14T07:09:10.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8979999999999999e-06 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112268+0000", 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7269999999999998e-06 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184668+0000", 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072399986999999999 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184737+0000", 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8813999999999995e-05 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184828+0000", 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0971999999999996e-05 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10174 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 522240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:10.973 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8147073049999998, 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072728104000000002, 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10174 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:10.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112277+0000", 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7100000000000002e-07 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112280+0000", 2024-09-14T07:09:10.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4310000000000001e-06 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112286+0000", 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4540000000000004e-06 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112291+0000", 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.989e-06 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184857+0000", 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072565397000000004 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.184922+0000", 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5866999999999998e-05 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185005+0000", 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2395e-05 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.977 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10175 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 544768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112299+0000", 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.81468476, 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072880912000000006, 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10175 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.978 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112299+0000", 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112299+0000", 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112300+0000", 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.63e-07 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112302+0000", 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.345e-06 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112309+0000", 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112314+0000", 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185033+0000", 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072719324000000002 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185102+0000", 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8714999999999995e-05 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185180+0000", 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8251999999999995e-05 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10177 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 567296~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.81466248, 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073321711999999997, 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10177 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112322+0000", 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7199999999999999e-07 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112324+0000", 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.277e-06 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112331+0000", 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3260000000000001e-06 2024-09-14T07:09:10.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112336+0000", 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.916e-06 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185383+0000", 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073047415000000004 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185490+0000", 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107361 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185643+0000", 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015294500000000001 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10179 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 589824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112367+0000", 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8146166350000001, 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073522753999999996, 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.986 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10179 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112367+0000", 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112367+0000", 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112368+0000", 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4799999999999998e-07 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112371+0000", 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.666e-06 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112387+0000", 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6413999999999999e-05 2024-09-14T07:09:10.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112393+0000", 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5029999999999997e-06 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185694+0000", 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073301253999999996 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185774+0000", 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0319000000000001e-05 2024-09-14T07:09:10.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185890+0000", 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011605 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10180 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 612352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112402+0000", 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.814582369, 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073678974999999994, 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.990 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10180 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112402+0000", 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112402+0000", 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112471+0000", 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9459e-05 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112477+0000", 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112486+0000", 2024-09-14T07:09:10.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9840000000000007e-06 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112492+0000", 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1179999999999997e-06 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185911+0000", 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073418330000000004 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.185974+0000", 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3585999999999997e-05 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186081+0000", 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010649099999999999 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10182 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 634880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112503+0000", 2024-09-14T07:09:10.994 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.814481083, 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073876662999999995, 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10182 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112503+0000", 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112503+0000", 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112842+0000", 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033877899999999999 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112847+0000", 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.904e-06 2024-09-14T07:09:10.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112858+0000", 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1610000000000001e-05 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112866+0000", 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8949999999999998e-06 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186123+0000", 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073257231000000006 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186253+0000", 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012930799999999999 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186380+0000", 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012693599999999999 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10183 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 657408~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112882+0000", 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8141016749999999, 2024-09-14T07:09:10.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073654547000000001, 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10183 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112882+0000", 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:10.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112882+0000", 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112883+0000", 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5400000000000001e-07 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112886+0000", 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9160000000000001e-06 2024-09-14T07:09:11.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112894+0000", 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7689999999999996e-06 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112900+0000", 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1630000000000001e-06 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186399+0000", 2024-09-14T07:09:11.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073499373000000007 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186463+0000", 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3675000000000003e-05 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186537+0000", 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3997000000000003e-05 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10185 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 679936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.112909+0000", 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8140746610000003, 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073873810999999998, 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10185 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112909+0000", 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.112909+0000", 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113242+0000", 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033220400000000001 2024-09-14T07:09:11.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113246+0000", 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7740000000000004e-06 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113257+0000", 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0749e-05 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113265+0000", 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4340000000000003e-06 2024-09-14T07:09:11.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186582+0000", 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073317462999999999 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186660+0000", 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8092999999999999e-05 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186783+0000", 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123094 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10187 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.113278+0000", 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8137060429999998, 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073745920000000006, 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.007 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10187 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113278+0000", 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113278+0000", 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113279+0000", 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3399999999999998e-07 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113282+0000", 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113289+0000", 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4909999999999999e-06 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113295+0000", 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9089999999999997e-06 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186834+0000", 2024-09-14T07:09:11.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073539123999999997 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.186913+0000", 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9289000000000006e-05 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187024+0000", 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110444 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10189 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 724992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.113305+0000", 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8136794389999999, 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073949126000000004, 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.012 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10189 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113305+0000", 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.113305+0000", 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114430+0000", 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011255099999999999 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114435+0000", 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114450+0000", 2024-09-14T07:09:11.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5452e-05 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114459+0000", 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1999999999999994e-06 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187068+0000", 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072609952000000005 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187149+0000", 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0675999999999998e-05 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187254+0000", 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010461199999999999 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10191 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.016 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.114473+0000", 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.812510815, 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073037823000000002, 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10191 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114473+0000", 2024-09-14T07:09:11.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114473+0000", 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114699+0000", 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022595199999999999 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114704+0000", 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114713+0000", 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2660000000000007e-06 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114721+0000", 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0949999999999998e-06 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187303+0000", 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072582038000000001 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187392+0000", 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9924000000000002e-05 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187511+0000", 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011858699999999999 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.020 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10193 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 770048~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.114734+0000", 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8122502420000002, 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.073012056000000006, 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.021 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10193 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114734+0000", 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.114734+0000", 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115076+0000", 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034258599999999998 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115099+0000", 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2130999999999999e-05 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115109+0000", 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0008e-05 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115116+0000", 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6650000000000003e-06 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187548+0000", 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072432283 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187632+0000", 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3380999999999993e-05 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187746+0000", 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114002 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:10195 3.1f 3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:14 [read 792576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:07.115130+0000", 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.8118537670000001, 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072812720999999997, 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.025 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10195 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115130+0000", 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115130+0000", 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115699+0000", 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056919800000000003 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115705+0000", 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115714+0000", 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6339999999999996e-06 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.115721+0000", 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.17e-06 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187780+0000", 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.072058104999999997 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187848+0000", 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8683999999999995e-05 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:07.187943+0000", 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4757000000000001e-05 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.029 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:11.031 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:11.031 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:11.024+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:11.132 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:11.128+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:11.200 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:11.200 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:11.200 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:11.214 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:11.233 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:11.232+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:11.330 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:11.330 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:11.330 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:11.331 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:11.331 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:09:11.333 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:11.328+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:11.352 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:11.434 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:11.428+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:11.522 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 54.223019077000004, 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:11.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:11.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.363543489999998, 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:11.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:11.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.363441205000001, 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:11.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:11.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.664389364000002, 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:11.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:11.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.179468172, 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:11.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.179432952, 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:09:11.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:09:11.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:11.551 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1793778030000004, 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:11.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:09:11.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1793437339999997, 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:09:11.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1793003879999997, 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:09:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:09:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1792668649999998, 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.568 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1792196150000001, 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:11.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:09:11.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:09:11.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.179105367, 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:09:11.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1789956720000001, 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:09:11.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:09:11.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1789295290000004, 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.584 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1788130250000002, 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.585 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:09:11.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:09:11.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1787473689999999, 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:09:11.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:09:11.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1786880489999998, 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:09:11.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:09:11.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1785471809999999, 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1781320739999996, 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.601 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:11.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:09:11.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.1780643350000002, 2024-09-14T07:09:11.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:11.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:11.609 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:11.610 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:11.611 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:11.604+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:11.711 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:11.708+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:11.741 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:11.741 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:11.741 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:11.741 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:11.756 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:11.812 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:11.808+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:11.913 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:11.908+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:11.919 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:11.919 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:11.919 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:11.933 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:12.014 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:12.008+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:12.060 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:12.060 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:12.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:12.060 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.227360802999996, 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:12.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:12.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.226586455000003, 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.066 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:12.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.215403457000001, 2024-09-14T07:09:12.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:12.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:12.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:12.077 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.918475250999997, 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:12.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6835714270000004, 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:12.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.087 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6830096359999995, 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:09:12.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:09:12.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6825150689999999, 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:12.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.096 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6818976499999998, 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:09:12.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:09:12.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6814693910000003, 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.105 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6810735269999997, 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6804818509999997, 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:09:12.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6771527370000001, 2024-09-14T07:09:12.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:09:12.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6771116859999999, 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.119 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:09:12.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6770594259999996, 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:09:12.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6770205389999999, 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:09:12.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:09:12.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6769611959999997, 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.676907108, 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.136 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.676867755, 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:09:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6768299750000004, 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:09:12.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:12.149 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6767932720000003, 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:09:12.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:09:12.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:12.153 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:12.154 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:12.155 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:12.148+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:12.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:12.255 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:12.252+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:12.292 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:12.292 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:12.292 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:12.292 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:12.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:12.391 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:12.388+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:12.458 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:12.452+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:12.472 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:12.472 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:12.472 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:12.472 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:12.472 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:12.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:12.543 INFO:teuthology.orchestra.run.smithi145.stdout:2640 punch from 0x728a400 to 0x7296000, (0xbc00 bytes) 2024-09-14T07:09:12.565 INFO:teuthology.orchestra.run.smithi145.stdout:2641 read 0x3f0b800 thru 0x3f16da8 (0xb5a9 bytes) 2024-09-14T07:09:12.565 INFO:teuthology.orchestra.run.smithi145.stdout:2643 read 0x5edc400 thru 0x5ee6c4f (0xa850 bytes) 2024-09-14T07:09:12.566 INFO:teuthology.orchestra.run.smithi145.stdout:2644 write 0x91a8200 thru 0x91a9aa9 (0x18aa bytes) 2024-09-14T07:09:12.566 INFO:teuthology.orchestra.run.smithi145.stdout:2645 read 0x2a29400 thru 0x2a31c25 (0x8826 bytes) 2024-09-14T07:09:12.567 INFO:teuthology.orchestra.run.smithi145.stdout:2648 punch from 0x578c200 to 0x5792e00, (0x6c00 bytes) 2024-09-14T07:09:12.575 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:12.572+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:12.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:12.661 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:12.652+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:12.697 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.407691227999997, 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:12.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:12.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.378183597000003, 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.707 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.369536238000002, 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:12.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:12.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.703443898000003, 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.536714672000002, 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:12.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.354783157999996, 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:09:12.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.603713663999997, 2024-09-14T07:09:12.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:12.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4732.0:183 3.1b 3:d96d4dd6:::rbd_data.1161bf39a192.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 6709248~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.753843623999998, 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030801473999999999, 2024-09-14T07:09:12.729 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4732", 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/303097868", 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 183 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942590+0000", 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944540+0000", 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001949824 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944564+0000", 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4199e-05 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944574+0000", 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944644+0000", 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0240999999999994e-05 2024-09-14T07:09:12.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944798+0000", 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153432 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.945082+0000", 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028403899999999998 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.959401+0000", 2024-09-14T07:09:12.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014319159999999999 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973083+0000", 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013682016 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973261+0000", 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017752999999999999 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973390+0000", 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012949500000000001 2024-09-14T07:09:12.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:538 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.762157994999999, 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020569724000000001, 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 538 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934275+0000", 2024-09-14T07:09:12.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934281+0000", 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934299+0000", 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8470000000000001e-05 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934313+0000", 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3196e-05 2024-09-14T07:09:12.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954450+0000", 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020137616000000001 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954629+0000", 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017857499999999999 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954844+0000", 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021515099999999999 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.738 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:539 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.762036245000001, 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02104733, 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 539 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.739 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934398+0000", 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.813e-06 2024-09-14T07:09:12.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934404+0000", 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7460000000000003e-06 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934425+0000", 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.033e-05 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934440+0000", 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5019e-05 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954916+0000", 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020476173 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955086+0000", 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169655 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955443+0000", 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035759399999999998 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.742 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:540 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.76196865, 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021617992999999999, 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 540 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.743 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934465+0000", 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4130000000000001e-06 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934471+0000", 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7259999999999999e-06 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934489+0000", 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8263000000000001e-05 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934506+0000", 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955539+0000", 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021033046999999999 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955813+0000", 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00027412200000000002 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956082+0000", 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000268829 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:541 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761903711999999, 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022127142999999998, 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 541 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934530+0000", 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.037e-06 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934535+0000", 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8259999999999998e-06 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934554+0000", 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934567+0000", 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3573e-05 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956183+0000", 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021615059999999998 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956395+0000", 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021209399999999999 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956656+0000", 2024-09-14T07:09:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026098000000000001 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:542 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761844720999999, 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022604783999999999, 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 542 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934589+0000", 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1039999999999999e-06 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934594+0000", 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.293e-06 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934612+0000", 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8578999999999999e-05 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934626+0000", 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3613e-05 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956738+0000", 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022111927 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956951+0000", 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021302100000000001 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957192+0000", 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024124699999999999 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:543 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761787178999999, 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025291999999999, 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 543 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934646+0000", 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934652+0000", 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000004e-06 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934670+0000", 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.808e-05 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934683+0000", 2024-09-14T07:09:12.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3686000000000001e-05 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957298+0000", 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02261471 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957448+0000", 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015033900000000001 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957670+0000", 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022180100000000001 2024-09-14T07:09:12.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:544 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761729553999999, 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023570109999999998, 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 544 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934704+0000", 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934726+0000", 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2005999999999999e-05 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934744+0000", 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8297000000000001e-05 2024-09-14T07:09:12.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934788+0000", 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3631999999999998e-05 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957813+0000", 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025417999999999 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958049+0000", 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023578 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958273+0000", 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022392700000000001 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:545 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761616372999999, 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023972594999999999, 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 545 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934817+0000", 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3060000000000001e-06 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934823+0000", 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-06 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934843+0000", 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9576e-05 2024-09-14T07:09:12.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934859+0000", 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.588e-05 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958368+0000", 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023509124999999999 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958487+0000", 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000119123 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958788+0000", 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030121500000000002 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:546 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761515606, 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024419039, 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 546 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:12.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934918+0000", 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934925+0000", 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9739999999999996e-06 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934946+0000", 2024-09-14T07:09:12.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0528999999999999e-05 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934960+0000", 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4343e-05 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958881+0000", 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023921114 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959100+0000", 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021872699999999999 2024-09-14T07:09:12.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959336+0000", 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023562600000000001 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:547 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761449297999999, 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024711538000000002, 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 547 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:12.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934984+0000", 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.513e-06 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934990+0000", 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5929999999999996e-06 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935008+0000", 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7802999999999999e-05 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935021+0000", 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3506000000000001e-05 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959426+0000", 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024404250999999998 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959538+0000", 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112534 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959694+0000", 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015633800000000001 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:548 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.761386927, 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024957958999999998, 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 548 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:12.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935046+0000", 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.147e-06 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935052+0000", 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6130000000000003e-06 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935071+0000", 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8522999999999999e-05 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935084+0000", 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3823e-05 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959754+0000", 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024669673 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959864+0000", 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109757 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.960003+0000", 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013942300000000001 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.779 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:591 3.1f 3:f9b02d91:::rbd_data.1161605482a1.0000000000000004:e [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.760432363, 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020703709000000001, 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 591 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:12.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936001+0000", 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.083e-06 2024-09-14T07:09:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936006+0000", 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936024+0000", 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7693000000000002e-05 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936037+0000", 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3303e-05 2024-09-14T07:09:12.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956267+0000", 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020229626000000001 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956482+0000", 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000215069 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956704+0000", 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022182500000000001 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.783 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:12.784 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:12.784 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:12.784 INFO:teuthology.orchestra.run.smithi145.stdout:2649 trunc from 0xa04d200 to 0xc1ae800 2024-09-14T07:09:12.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:12.789 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:12.780+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:12.798 INFO:teuthology.orchestra.run.smithi145.stdout:2651 read 0x3d3e00 thru 0x3dfac6 (0xbcc7 bytes) 2024-09-14T07:09:12.805 INFO:teuthology.orchestra.run.smithi145.stdout:2652 read 0xa65e000 thru 0xa66b0cb (0xd0cc bytes) 2024-09-14T07:09:12.805 INFO:teuthology.orchestra.run.smithi145.stdout:2654 read 0x617ae00 thru 0x61807b2 (0x59b3 bytes) 2024-09-14T07:09:12.806 INFO:teuthology.orchestra.run.smithi145.stdout:2655 read 0x3fee00 thru 0x40d733 (0xe934 bytes) 2024-09-14T07:09:12.807 INFO:teuthology.orchestra.run.smithi145.stdout:2657 write 0x16f0200 thru 0x16fa154 (0x9f55 bytes) 2024-09-14T07:09:12.809 INFO:teuthology.orchestra.run.smithi145.stdout:2659 write 0x7fd7e00 thru 0x7fde70f (0x6910 bytes) 2024-09-14T07:09:12.810 INFO:teuthology.orchestra.run.smithi145.stdout:2660 write 0xb797800 thru 0xb79afe2 (0x37e3 bytes) 2024-09-14T07:09:12.810 INFO:teuthology.orchestra.run.smithi145.stdout:2661 read 0x25c0200 thru 0x25ca9bc (0xa7bd bytes) 2024-09-14T07:09:12.810 INFO:teuthology.orchestra.run.smithi145.stdout:2664 write 0xb8be800 thru 0xb8cca41 (0xe242 bytes) 2024-09-14T07:09:12.811 INFO:teuthology.orchestra.run.smithi145.stdout:2665 read 0xb041000 thru 0xb04d390 (0xc391 bytes) 2024-09-14T07:09:12.811 INFO:teuthology.orchestra.run.smithi145.stdout:2666 write 0x1efd200 thru 0x1f0766b (0xa46c bytes) 2024-09-14T07:09:12.812 INFO:teuthology.orchestra.run.smithi145.stdout:2669 punch from 0x2b4fc00 to 0x2b51a00, (0x1e00 bytes) 2024-09-14T07:09:12.837 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:12.864 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:12.885 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:12.880+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:12.887 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:12.915 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:12.949 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:12.972 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:12.972 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:12.981 INFO:teuthology.orchestra.run.smithi145.stdout:2674 read 0x66dac00 thru 0x66e2a91 (0x7e92 bytes) 2024-09-14T07:09:12.981 INFO:teuthology.orchestra.run.smithi145.stdout:2676 read 0x1742200 thru 0x17494fc (0x72fd bytes) 2024-09-14T07:09:12.988 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:12.980+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:12.991 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:12.991 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:13.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:13.030 INFO:teuthology.orchestra.run.smithi145.stdout:2677 write 0x7c6200 thru 0x7d1e2d (0xbc2e bytes) 2024-09-14T07:09:13.032 INFO:teuthology.orchestra.run.smithi145.stdout:2679 write 0x25a200 thru 0x264f52 (0xad53 bytes) 2024-09-14T07:09:13.034 INFO:teuthology.orchestra.run.smithi145.stdout:2681 read 0xe6c600 thru 0xe71671 (0x5072 bytes) 2024-09-14T07:09:13.035 INFO:teuthology.orchestra.run.smithi145.stdout:2684 read 0xbb10400 thru 0xbb13fc9 (0x3bca bytes) 2024-09-14T07:09:13.088 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:13.084+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:13.175 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:13.175 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:13.188 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:13.184+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:13.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:13.288 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:13.284+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:13.322 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:13.323 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:13.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:13.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:09:13.390 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:13.384+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:13.490 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:13.484+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:13.520 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:13.520 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:13.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:13.627 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:13.624+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:13.692 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:13.688+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:13.694 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:13.694 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:13.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:09:13.793 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:13.788+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:13.824 INFO:teuthology.orchestra.run.smithi145.stdout:2685 punch from 0x8a1ec00 to 0x8a21000, (0x2400 bytes) 2024-09-14T07:09:13.884 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:13.884 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:13.889 INFO:teuthology.orchestra.run.smithi145.stdout:2686 read 0x75ff400 thru 0x76058ab (0x64ac bytes) 2024-09-14T07:09:13.890 INFO:teuthology.orchestra.run.smithi145.stdout:2687 read 0x9cca600 thru 0x9cd12d4 (0x6cd5 bytes) 2024-09-14T07:09:13.893 INFO:teuthology.orchestra.run.smithi145.stdout:2689 read 0x5d4de00 thru 0x5d58199 (0xa39a bytes) 2024-09-14T07:09:13.894 INFO:teuthology.orchestra.run.smithi145.stdout:2691 write 0x4c99a00 thru 0x4ca5e65 (0xc466 bytes) 2024-09-14T07:09:13.894 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:13.888+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:13.894 INFO:teuthology.orchestra.run.smithi145.stdout:2692 read 0x9670e00 thru 0x967d773 (0xc974 bytes) 2024-09-14T07:09:13.895 INFO:teuthology.orchestra.run.smithi145.stdout:2693 write 0x1792e00 thru 0x17a01d6 (0xd3d7 bytes) 2024-09-14T07:09:13.896 INFO:teuthology.orchestra.run.smithi145.stdout:2695 read 0xb7f6000 thru 0xb7fde33 (0x7e34 bytes) 2024-09-14T07:09:13.896 INFO:teuthology.orchestra.run.smithi145.stdout:2698 write 0xa545600 thru 0xa5469e3 (0x13e4 bytes) 2024-09-14T07:09:13.899 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:13.899 INFO:teuthology.orchestra.run.smithi145.stdout:2699 read 0x3b54200 thru 0x3b55141 (0xf42 bytes) 2024-09-14T07:09:13.900 INFO:teuthology.orchestra.run.smithi145.stdout:2700 write 0x3760600 thru 0x376caa2 (0xc4a3 bytes) 2024-09-14T07:09:13.932 INFO:teuthology.orchestra.run.smithi145.stdout:2701 trunc from 0xc1ae800 to 0x1905c00 2024-09-14T07:09:13.994 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:13.988+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:14.034 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:14.034 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:14.048 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:14.139 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:14.136+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:14.196 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:14.192+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:14.216 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:14.216 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:14.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:14.296 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:14.296+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:14.395 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:14.396 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:14.397 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:14.392+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:14.410 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:14.498 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:14.492+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:14.534 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:14.534 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:14.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:14.599 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:14.592+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:14.672 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:14.672 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:14.686 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:14.728 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:14.724+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:14.801 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:14.796+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:14.875 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:14.875 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:14.889 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:14.902 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:14.896+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:14.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:15.003 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:14.996+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:15.091 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:15.091 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:15.104 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:15.100+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:15.105 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:15.150 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:15.150 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:15.166 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:15.167 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:15.185 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:15.203 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:15.203 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:15.205 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:15.200+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:15.209 INFO:teuthology.orchestra.run.smithi145.stdout:2702 trunc from 0x1905c00 to 0x71c800 2024-09-14T07:09:15.222 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:15.223 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:15.224 INFO:teuthology.orchestra.run.smithi145.stdout:2703 write 0x61000 thru 0x6a389 (0x938a bytes) 2024-09-14T07:09:15.227 INFO:teuthology.orchestra.run.smithi145.stdout:2706 punch from 0x4fac00 to 0x507200, (0xc600 bytes) 2024-09-14T07:09:15.241 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:15.241 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:15.253 INFO:teuthology.orchestra.run.smithi145.stdout:2707 write 0x65b9e00 thru 0x65c673d (0xc93e bytes) 2024-09-14T07:09:15.256 INFO:teuthology.orchestra.run.smithi145.stdout:2708 write 0x1645a00 thru 0x164c70f (0x6d10 bytes) 2024-09-14T07:09:15.257 INFO:teuthology.orchestra.run.smithi145.stdout:2709 punch from 0x5cd7a00 to 0x5ce3800, (0xbe00 bytes) 2024-09-14T07:09:15.257 INFO:teuthology.orchestra.run.smithi145.stdout:2710 read 0x3f69600 thru 0x3f71a86 (0x8487 bytes) 2024-09-14T07:09:15.271 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:15.271 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:15.272 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:15.272 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:15.281 INFO:teuthology.orchestra.run.smithi145.stdout:2714 write 0xbe01000 thru 0xbe0cd6b (0xbd6c bytes) 2024-09-14T07:09:15.282 INFO:teuthology.orchestra.run.smithi145.stdout:2715 read 0xa5d4200 thru 0xa5d8935 (0x4736 bytes) 2024-09-14T07:09:15.284 INFO:teuthology.orchestra.run.smithi145.stdout:2717 write 0x6f4a00 thru 0x704540 (0xfb41 bytes) 2024-09-14T07:09:15.285 INFO:teuthology.orchestra.run.smithi145.stdout:2718 write 0x3441800 thru 0x344c00e (0xa80f bytes) 2024-09-14T07:09:15.286 INFO:teuthology.orchestra.run.smithi145.stdout:2719 punch from 0x5ee6600 to 0x5ef0e00, (0xa800 bytes) 2024-09-14T07:09:15.286 INFO:teuthology.orchestra.run.smithi145.stdout:2721 punch from 0x18c5c00 to 0x18cee00, (0x9200 bytes) 2024-09-14T07:09:15.287 INFO:teuthology.orchestra.run.smithi145.stdout:2722 write 0x65b8400 thru 0x65bc2ef (0x3ef0 bytes) 2024-09-14T07:09:15.288 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:15.288 INFO:teuthology.orchestra.run.smithi145.stdout:2726 read 0x1c00400 thru 0x1c07527 (0x7128 bytes) 2024-09-14T07:09:15.290 INFO:teuthology.orchestra.run.smithi145.stdout:2729 read 0x8afb800 thru 0x8b00ff5 (0x57f6 bytes) 2024-09-14T07:09:15.291 INFO:teuthology.orchestra.run.smithi145.stdout:2730 read 0x9515c00 thru 0x951ee5f (0x9260 bytes) 2024-09-14T07:09:15.291 INFO:teuthology.orchestra.run.smithi145.stdout:2731 punch from 0xad71600 to 0xad72c00, (0x1600 bytes) 2024-09-14T07:09:15.292 INFO:teuthology.orchestra.run.smithi145.stdout:2732 write 0x4391600 thru 0x4397b0a (0x650b bytes) 2024-09-14T07:09:15.294 INFO:teuthology.orchestra.run.smithi145.stdout:2734 write 0x4c80000 thru 0x4c828fa (0x28fb bytes) 2024-09-14T07:09:15.295 INFO:teuthology.orchestra.run.smithi145.stdout:2735 read 0x2196200 thru 0x219c314 (0x6115 bytes) 2024-09-14T07:09:15.296 INFO:teuthology.orchestra.run.smithi145.stdout:2736 punch from 0x958c600 to 0x9597c00, (0xb600 bytes) 2024-09-14T07:09:15.310 INFO:teuthology.orchestra.run.smithi145.stdout:2742 trunc from 0xbe0ce00 to 0x3b9e200 2024-09-14T07:09:15.331 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:15.328+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:15.358 INFO:teuthology.orchestra.run.smithi145.stdout:2743 write 0x674f000 thru 0x67528a1 (0x38a2 bytes) 2024-09-14T07:09:15.360 INFO:teuthology.orchestra.run.smithi145.stdout:2746 read 0x674e200 thru 0x674ff40 (0x1d41 bytes) 2024-09-14T07:09:15.360 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:09:15.360 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:15.361 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T07:09:15.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:09:15.384 INFO:teuthology.orchestra.run.smithi145.stdout:2748 write 0x9c41600 thru 0x9c4302b (0x1a2c bytes) 2024-09-14T07:09:15.401 INFO:teuthology.orchestra.run.smithi145.stdout:2749 trunc from 0x9c43200 to 0x3eeac00 2024-09-14T07:09:15.407 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:15.404+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:15.425 INFO:teuthology.orchestra.run.smithi145.stdout:2750 flatten 2024-09-14T07:09:15.438 INFO:teuthology.orchestra.run.smithi145.stdout:2751 read 0x323be00 thru 0x3240395 (0x4596 bytes) 2024-09-14T07:09:15.440 INFO:teuthology.orchestra.run.smithi145.stdout:2754 read 0x222de00 thru 0x2231b40 (0x3d41 bytes) 2024-09-14T07:09:15.441 INFO:teuthology.orchestra.run.smithi145.stdout:2756 punch from 0x337c00 to 0x347a00, (0xfe00 bytes) 2024-09-14T07:09:15.441 INFO:teuthology.orchestra.run.smithi145.stdout:2757 read 0xfb5000 thru 0xfbf94e (0xa94f bytes) 2024-09-14T07:09:15.464 INFO:teuthology.orchestra.run.smithi145.stdout:2758 trunc from 0x3eeac00 to 0xd868a00 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 82.636971553999999, 2024-09-14T07:09:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 82.636197206000006, 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 53.625014208000003, 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 53.328086001999999, 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0931821779999993, 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0926203870000002, 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:09:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:09:15.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.504 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0921258199999997, 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:09:15.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0915084010000005, 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:09:15.509 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:15.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:09:15.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.514 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0910801419999991, 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.515 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:09:15.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.519 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0906842779999995, 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.520 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:09:15.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:09:15.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.525 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0900926020000004, 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:09:15.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:09:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.530 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0867634880000008, 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.531 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:09:15.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:15.535 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0867224370000006, 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:09:15.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.540 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0866701770000002, 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:09:15.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.545 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0866312899999997, 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:09:15.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.550 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0865719469999995, 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.551 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:09:15.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.555 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0865178590000006, 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:09:15.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:09:15.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0864785060000006, 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:09:15.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0864407259999993, 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.566 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:09:15.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0864040230000001, 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:15.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:15.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:09:15.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:15.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:09:15.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:09:15.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.576 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:15.576 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:15.576 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:15.579 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:15.572+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:15.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:15.581 INFO:teuthology.orchestra.run.smithi145.stdout:2759 write 0xba3fc00 thru 0xba4bc62 (0xc063 bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2760 read 0x658e600 thru 0x6594c06 (0x6607 bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2761 read 0x4ef1600 thru 0x4ef9239 (0x7c3a bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2763 punch from 0x5d63e00 to 0x5d6aa00, (0x6c00 bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2765 read 0x4936400 thru 0x494150c (0xb10d bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2767 write 0x501e00 thru 0x507f2c (0x612d bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2768 punch from 0x14d400 to 0x14da00, (0x600 bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2769 punch from 0xcc9dc00 to 0xcca0c00, (0x3000 bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2772 trunc from 0xd868a00 to 0xa3c9800 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2773 read 0x2355800 thru 0x23592dc (0x3add bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2774 write 0x3c9e00 thru 0x3cf4de (0x56df bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2775 read 0x678d800 thru 0x679354c (0x5d4d bytes) 2024-09-14T07:09:15.582 INFO:teuthology.orchestra.run.smithi145.stdout:2778 write 0x6c19400 thru 0x6c1e3a7 (0x4fa8 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2782 write 0x2e3ee00 thru 0x2e4b50a (0xc70b bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2783 write 0x9f3c400 thru 0x9f406a7 (0x42a8 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2784 write 0xe3da600 thru 0xe3de091 (0x3a92 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2786 write 0xad5fc00 thru 0xad63270 (0x3671 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2787 read 0x8b21e00 thru 0x8b2ba8b (0x9c8c bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2788 punch from 0xa2d000 to 0xa32200, (0x5200 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2792 read 0x7b38200 thru 0x7b39c71 (0x1a72 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2793 read 0x6595e00 thru 0x6596b60 (0xd61 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2794 write 0x574f600 thru 0x575aa66 (0xb467 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2801 read 0x1153200 thru 0x1157c96 (0x4a97 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2802 write 0x20dde00 thru 0x20ed3b7 (0xf5b8 bytes) 2024-09-14T07:09:15.583 INFO:teuthology.orchestra.run.smithi145.stdout:2803 punch from 0x8a52e00 to 0x8a5aa00, (0x7c00 bytes) 2024-09-14T07:09:15.584 INFO:teuthology.orchestra.run.smithi145.stdout:2807 write 0x51b4e00 thru 0x51c4608 (0xf809 bytes) 2024-09-14T07:09:15.584 INFO:teuthology.orchestra.run.smithi145.stdout:2809 clone 20 order 25 su 4194304 sc 3 2024-09-14T07:09:15.678 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:15.676+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:15.698 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 89, 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:09:14.764447+0000", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:09:10.322879+0000", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5806080, 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "noscrub", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:09:15.699 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "luminous", 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:09:15.700 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:09:15.701 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:09:15.702 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:09:15.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:09:15.704 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:09:15.705 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 6.059999942779541, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 6.059999942779541, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.33000001311302185, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:09:15.706 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:09:15.707 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:09:15.708 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:09:15.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:09:15.710 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:09:15.711 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 2.25, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2.25, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:09:15.712 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:09:15.713 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:09:15.714 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "83", 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 21, 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 83, 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:09:15.715 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:09:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:09:15.717 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.690000057220459, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.690000057220459, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.690000057220459, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.690000057220459, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:09:15.718 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.297698974609375, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:15.719 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.720 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:09:15.723 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.724 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:15.725 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:09:15.729 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:15.730 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.732 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.733 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:09:15.734 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:09:15.735 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.737 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.738 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.739 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.740 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:15.741 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:15.746 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:15.747 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.750 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.751 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.752 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:15.753 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:15.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [ 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: "pgid": "3.13", 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: "mappings": [ 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: "from": 3, 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: "to": 1 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:15.757 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-09-14T07:09:15.758 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:15.759 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:15.760 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:15.760 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:15.760 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:15.760 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:15.761 INFO:tasks.thrashosds.thrasher:j is {'epoch': 89, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:09:14.764447+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:09:10.322879+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.297698974609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.13', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:09:15.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:15.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:15.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:09:15.812 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:15.808+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:15.880 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:15.876+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:15.934 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:15.934 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:15.934 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:15.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:16.003 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:16.000+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:16.082 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:16.080+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:16.120 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.831147588, 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:16.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:16.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.801639956999999, 2024-09-14T07:09:16.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:16.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.792992597999998, 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 54.126900257999999, 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:16.141 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.960171031999998, 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:16.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:16.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:16.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:1024 3.15 e46/21 3:af60922b:::rbd_data.1161ff1af5c4.0000000000000009:head v 46'73, mlcod=43'70)", 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.778239517999999, 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020654333, 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341649+0000", 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.341651+0000", 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.652e-06 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346278+0000", 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046270449999999998 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346310+0000", 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2283999999999998e-05 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346324+0000", 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4473e-05 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346416+0000", 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1218000000000007e-05 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.346444+0000", 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8603999999999999e-05 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362265+0000", 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015821176999999999 2024-09-14T07:09:16.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.362303+0000", 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7880000000000003e-05 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.027170024, 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:16.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:16.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:16.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:16.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:16.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:16.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:16.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:16.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:16.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:16.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:16.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:16.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:16.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:16.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:16.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:16.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:16.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.159 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.160 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4732.0:183 3.1b 3:d96d4dd6:::rbd_data.1161bf39a192.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 6709248~29696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-14T07:09:16.160 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:16.160 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.177299984000001, 2024-09-14T07:09:16.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030801473999999999, 2024-09-14T07:09:16.160 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4732", 2024-09-14T07:09:16.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/303097868", 2024-09-14T07:09:16.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 183 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942589+0000", 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.942590+0000", 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944540+0000", 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001949824 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944564+0000", 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4199e-05 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944574+0000", 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944644+0000", 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0240999999999994e-05 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.944798+0000", 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000153432 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.945082+0000", 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028403899999999998 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.959401+0000", 2024-09-14T07:09:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014319159999999999 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973083+0000", 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013682016 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973261+0000", 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017752999999999999 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:29.973390+0000", 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012949500000000001 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:538 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 5767168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185614354999998, 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020569724000000001, 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 538 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934274+0000", 2024-09-14T07:09:16.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934275+0000", 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934281+0000", 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934299+0000", 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8470000000000001e-05 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934313+0000", 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3196e-05 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954450+0000", 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020137616000000001 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954629+0000", 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017857499999999999 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954844+0000", 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021515099999999999 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:539 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 6750208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185492605, 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02104733, 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 539 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934396+0000", 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934398+0000", 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.813e-06 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934404+0000", 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7460000000000003e-06 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934425+0000", 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.033e-05 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934440+0000", 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5019e-05 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.954916+0000", 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020476173 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955086+0000", 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169655 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955443+0000", 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035759399999999998 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:540 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 7733248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185425010000003, 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021617992999999999, 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 540 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934464+0000", 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934465+0000", 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4130000000000001e-06 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934471+0000", 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7259999999999999e-06 2024-09-14T07:09:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934489+0000", 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8263000000000001e-05 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934506+0000", 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955539+0000", 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021033046999999999 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.955813+0000", 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00027412200000000002 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956082+0000", 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000268829 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:541 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 9306112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185360072000002, 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022127142999999998, 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 541 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934528+0000", 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934530+0000", 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.037e-06 2024-09-14T07:09:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934535+0000", 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8259999999999998e-06 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934554+0000", 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934567+0000", 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3573e-05 2024-09-14T07:09:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956183+0000", 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021615059999999998 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956395+0000", 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021209399999999999 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956656+0000", 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026098000000000001 2024-09-14T07:09:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:542 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 10289152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185301080999999, 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022604783999999999, 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 542 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934587+0000", 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934589+0000", 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1039999999999999e-06 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934594+0000", 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.293e-06 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934612+0000", 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8578999999999999e-05 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934626+0000", 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3613e-05 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956738+0000", 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022111927 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956951+0000", 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021302100000000001 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957192+0000", 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024124699999999999 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:543 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 11272192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185243538999998, 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025291999999999, 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 543 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934645+0000", 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934646+0000", 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934652+0000", 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4790000000000004e-06 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934670+0000", 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.808e-05 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934683+0000", 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3686000000000001e-05 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957298+0000", 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02261471 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957448+0000", 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015033900000000001 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957670+0000", 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022180100000000001 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:544 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12255232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185185914000002, 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023570109999999998, 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 544 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934703+0000", 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934704+0000", 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934726+0000", 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2005999999999999e-05 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934744+0000", 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8297000000000001e-05 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934788+0000", 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3631999999999998e-05 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.957813+0000", 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023025417999999999 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958049+0000", 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023578 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958273+0000", 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022392700000000001 2024-09-14T07:09:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:545 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 12845056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.185072732999998, 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023972594999999999, 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 545 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934816+0000", 2024-09-14T07:09:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934817+0000", 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3060000000000001e-06 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934823+0000", 2024-09-14T07:09:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-06 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934843+0000", 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9576e-05 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934859+0000", 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.588e-05 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958368+0000", 2024-09-14T07:09:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023509124999999999 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958487+0000", 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000119123 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958788+0000", 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030121500000000002 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:546 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.184971965999999, 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024419039, 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 546 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934917+0000", 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934918+0000", 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934925+0000", 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9739999999999996e-06 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934946+0000", 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0528999999999999e-05 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934960+0000", 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4343e-05 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.958881+0000", 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023921114 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959100+0000", 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021872699999999999 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959336+0000", 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00023562600000000001 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:547 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 14811136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.184905657999998, 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024711538000000002, 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 547 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934983+0000", 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934984+0000", 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.513e-06 2024-09-14T07:09:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.934990+0000", 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5929999999999996e-06 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935008+0000", 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7802999999999999e-05 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935021+0000", 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3506000000000001e-05 2024-09-14T07:09:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959426+0000", 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024404250999999998 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959538+0000", 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112534 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959694+0000", 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015633800000000001 2024-09-14T07:09:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:548 3.1b 3:dbc3c005:::rbd_data.1161605482a1.0000000000000001:e [sparse-read 15794176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.184843287, 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024957958999999998, 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 548 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935045+0000", 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935046+0000", 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.147e-06 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935052+0000", 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6130000000000003e-06 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935071+0000", 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8522999999999999e-05 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.935084+0000", 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3823e-05 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959754+0000", 2024-09-14T07:09:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024669673 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.959864+0000", 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109757 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.960003+0000", 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013942300000000001 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4867.0:591 3.1f 3:f9b02d91:::rbd_data.1161605482a1.0000000000000004:e [sparse-read 12451840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.183888723000003, 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020703709000000001, 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4867", 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2785074862", 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 591 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936000+0000", 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936001+0000", 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.083e-06 2024-09-14T07:09:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936006+0000", 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936024+0000", 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7693000000000002e-05 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.936037+0000", 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3303e-05 2024-09-14T07:09:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956267+0000", 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020229626000000001 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956482+0000", 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000215069 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:43.956704+0000", 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00022182500000000001 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.220 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:16.222 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:16.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:16.224 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-14T07:09:15.786327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1298006667,"num_objects":601,"num_object_clones":87,"num_object_copies":1202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":601,"num_whiteouts":175,"num_read":118591,"num_read_kb":538386,"num_write":7621,"num_write_kb":209796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":971,"num_bytes_recovered":128600551,"num_keys_recovered":276,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6166,"ondisk_log_size":6166,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":247104,"kb_used_data":34988,"kb_used_omap":9,"kb_used_meta":211958,"kb_avail":565983936,"statfs":{"total":579820584960,"available":579567550464,"internally_reserved":0,"allocated":35827712,"data_stored":337088563,"data_compressed":9505745,"data_compressed_allocated":25006080,"data_compressed_original":331836928,"omap_allocated":9568,"internal_metadata":217045664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28019712,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8878,"num_read_kb":21578,"num_write":100,"num_write_kb":13082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":39351176,"num_keys_recovered":67,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.450743"},"pg_stats":[{"pgid":"3.1f","version":"82'109","reported_seq":3858,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.123663+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:09:15.123663+0000","last_peered":"2024-09-14T07:09:15.123663+0000","last_clean":"2024-09-14T07:09:15.123663+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:09:15.123663+0000","last_undegraded":"2024-09-14T07:09:15.123663+0000","last_fullsized":"2024-09-14T07:09:15.123663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:03.040416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":68849680,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":3340,"num_read_kb":40972,"num_write":184,"num_write_kb":9642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"82'325","reported_seq":4917,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.986685+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:14.986685+0000","last_peered":"2024-09-14T07:09:14.986685+0000","last_clean":"2024-09-14T07:09:14.986685+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:14.986685+0000","last_undegraded":"2024-09-14T07:09:14.986685+0000","last_fullsized":"2024-09-14T07:09:14.986685+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:13.096701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":58980591,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":5585,"num_read_kb":36472,"num_write":550,"num_write_kb":12260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"83'300","reported_seq":4351,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.107470+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:15.107470+0000","last_peered":"2024-09-14T07:09:15.107470+0000","last_clean":"2024-09-14T07:09:15.107470+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:15.107470+0000","last_undegraded":"2024-09-14T07:09:15.107470+0000","last_fullsized":"2024-09-14T07:09:15.107470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:15:20.988318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":8611877,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":5560,"num_read_kb":13417,"num_write":232,"num_write_kb":2184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"83'105","reported_seq":1657,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.768891+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:14.768891+0000","last_peered":"2024-09-14T07:09:14.768891+0000","last_clean":"2024-09-14T07:09:14.768891+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:14.768891+0000","last_undegraded":"2024-09-14T07:09:14.768891+0000","last_fullsized":"2024-09-14T07:09:14.768891+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:08.751908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":22163508,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":1424,"num_read_kb":9021,"num_write":114,"num_write_kb":3375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'332","reported_seq":5866,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.274809+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:09:15.274809+0000","last_peered":"2024-09-14T07:09:15.274809+0000","last_clean":"2024-09-14T07:09:15.274809+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:09:15.274809+0000","last_undegraded":"2024-09-14T07:09:15.274809+0000","last_fullsized":"2024-09-14T07:09:15.274809+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:04:56.041384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":94885563,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":6322,"num_read_kb":15558,"num_write":458,"num_write_kb":9026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"82'151","reported_seq":6503,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T07:09:11.338713+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:11.338713+0000","last_peered":"2024-09-14T07:09:11.338713+0000","last_clean":"2024-09-14T07:09:11.338713+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:11.338713+0000","last_undegraded":"2024-09-14T07:09:11.338713+0000","last_fullsized":"2024-09-14T07:09:11.338713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:11.004691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":69664903,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":7209,"num_read_kb":14540,"num_write":187,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"89'116","reported_seq":1597,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.182739+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:15.182739+0000","last_peered":"2024-09-14T07:09:15.182739+0000","last_clean":"2024-09-14T07:09:15.182739+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:15.182739+0000","last_undegraded":"2024-09-14T07:09:15.182739+0000","last_fullsized":"2024-09-14T07:09:15.182739+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:29:54.052528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":22634603,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1444,"num_read_kb":9998,"num_write":136,"num_write_kb":5279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"89'134","reported_seq":1957,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.265467+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:09:15.265467+0000","last_peered":"2024-09-14T07:09:15.265467+0000","last_clean":"2024-09-14T07:09:15.265467+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:09:15.265467+0000","last_undegraded":"2024-09-14T07:09:15.265467+0000","last_fullsized":"2024-09-14T07:09:15.265467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:47.537967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":70687792,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":1653,"num_read_kb":4984,"num_write":255,"num_write_kb":13273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"89'73","reported_seq":1200,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.261415+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:09:15.261415+0000","last_peered":"2024-09-14T07:09:15.261415+0000","last_clean":"2024-09-14T07:09:15.261415+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:09:15.261415+0000","last_undegraded":"2024-09-14T07:09:15.261415+0000","last_fullsized":"2024-09-14T07:09:15.261415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:54:53.049081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":12451878,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":835,"num_read_kb":7736,"num_write":80,"num_write_kb":2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"82'160","reported_seq":4919,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.151806+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:15.151806+0000","last_peered":"2024-09-14T07:09:15.151806+0000","last_clean":"2024-09-14T07:09:15.151806+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:15.151806+0000","last_undegraded":"2024-09-14T07:09:15.151806+0000","last_fullsized":"2024-09-14T07:09:15.151806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:07:57.327370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":47148070,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":4788,"num_read_kb":27801,"num_write":158,"num_write_kb":3788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'132","reported_seq":2988,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.106545+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:09:15.106545+0000","last_peered":"2024-09-14T07:09:15.106545+0000","last_clean":"2024-09-14T07:09:15.106545+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:09:15.106545+0000","last_undegraded":"2024-09-14T07:09:15.106545+0000","last_fullsized":"2024-09-14T07:09:15.106545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:11:55.607448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":63338545,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":2864,"num_read_kb":34933,"num_write":218,"num_write_kb":12812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"83'156","reported_seq":4595,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.986894+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:14.986894+0000","last_peered":"2024-09-14T07:09:14.986894+0000","last_clean":"2024-09-14T07:09:14.986894+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:14.986894+0000","last_undegraded":"2024-09-14T07:09:14.986894+0000","last_fullsized":"2024-09-14T07:09:14.986894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:36:21.084811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":53785104,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":4687,"num_read_kb":18666,"num_write":126,"num_write_kb":3307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.773102+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:14.773102+0000","last_peered":"2024-09-14T07:09:14.773102+0000","last_clean":"2024-09-14T07:09:14.773102+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:14.773102+0000","last_undegraded":"2024-09-14T07:09:14.773102+0000","last_fullsized":"2024-09-14T07:09:14.773102+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"83'306","reported_seq":4145,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.131726+0000","last_change":"2024-09-14T07:09:14.793462+0000","last_active":"2024-09-14T07:09:15.131726+0000","last_peered":"2024-09-14T07:09:15.131726+0000","last_clean":"2024-09-14T07:09:15.131726+0000","last_became_active":"2024-09-14T07:09:14.787124+0000","last_became_peered":"2024-09-14T07:09:14.787124+0000","last_unstale":"2024-09-14T07:09:15.131726+0000","last_undegraded":"2024-09-14T07:09:15.131726+0000","last_fullsized":"2024-09-14T07:09:15.131726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:16:21.316315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":14473449,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":5957,"num_read_kb":15518,"num_write":327,"num_write_kb":2081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1461720,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.781641+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:14.781641+0000","last_peered":"2024-09-14T07:09:14.781641+0000","last_clean":"2024-09-14T07:09:14.781641+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:14.781641+0000","last_undegraded":"2024-09-14T07:09:14.781641+0000","last_fullsized":"2024-09-14T07:09:14.781641+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"80'173","reported_seq":3629,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.015790+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:15.015790+0000","last_peered":"2024-09-14T07:09:15.015790+0000","last_clean":"2024-09-14T07:09:15.015790+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:15.015790+0000","last_undegraded":"2024-09-14T07:09:15.015790+0000","last_fullsized":"2024-09-14T07:09:15.015790+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:00:28.344378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":33835038,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":4035,"num_read_kb":12630,"num_write":209,"num_write_kb":2086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.772638+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:09:14.772638+0000","last_peered":"2024-09-14T07:09:14.772638+0000","last_clean":"2024-09-14T07:09:14.772638+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:09:14.772638+0000","last_undegraded":"2024-09-14T07:09:14.772638+0000","last_fullsized":"2024-09-14T07:09:14.772638+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"82'425","reported_seq":2088,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.781532+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:14.781532+0000","last_peered":"2024-09-14T07:09:14.781532+0000","last_clean":"2024-09-14T07:09:14.781532+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:14.781532+0000","last_undegraded":"2024-09-14T07:09:14.781532+0000","last_fullsized":"2024-09-14T07:09:14.781532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:53.115264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":393271,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":3869,"num_read_kb":3707,"num_write":631,"num_write_kb":1502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.781795+0000","last_change":"2024-09-14T07:09:11.349379+0000","last_active":"2024-09-14T07:09:14.781795+0000","last_peered":"2024-09-14T07:09:14.781795+0000","last_clean":"2024-09-14T07:09:14.781795+0000","last_became_active":"2024-09-14T07:09:11.348941+0000","last_became_peered":"2024-09-14T07:09:11.348941+0000","last_unstale":"2024-09-14T07:09:14.781795+0000","last_undegraded":"2024-09-14T07:09:14.781795+0000","last_fullsized":"2024-09-14T07:09:14.781795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:39.931750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"89'70","reported_seq":1997,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.249191+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:15.249191+0000","last_peered":"2024-09-14T07:09:15.249191+0000","last_clean":"2024-09-14T07:09:15.249191+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:15.249191+0000","last_undegraded":"2024-09-14T07:09:15.249191+0000","last_fullsized":"2024-09-14T07:09:15.249191+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:16:22.618417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":8532038,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":1789,"num_read_kb":14012,"num_write":95,"num_write_kb":3500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'65","reported_seq":2331,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T07:09:11.678902+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:09:11.678902+0000","last_peered":"2024-09-14T07:09:11.678902+0000","last_clean":"2024-09-14T07:09:11.678902+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:09:11.678902+0000","last_undegraded":"2024-09-14T07:09:11.678902+0000","last_fullsized":"2024-09-14T07:09:11.678902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:42:57.710290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":45023232,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":1761,"num_read_kb":9425,"num_write":141,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":162,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.768761+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:14.768761+0000","last_peered":"2024-09-14T07:09:14.768761+0000","last_clean":"2024-09-14T07:09:14.768761+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:14.768761+0000","last_undegraded":"2024-09-14T07:09:14.768761+0000","last_fullsized":"2024-09-14T07:09:14.768761+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"89'226","reported_seq":2638,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.184385+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:15.184385+0000","last_peered":"2024-09-14T07:09:15.184385+0000","last_clean":"2024-09-14T07:09:15.184385+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:15.184385+0000","last_undegraded":"2024-09-14T07:09:15.184385+0000","last_fullsized":"2024-09-14T07:09:15.184385+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:33:50.498868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":3689159,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":4463,"num_read_kb":7052,"num_write":147,"num_write_kb":1757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.775726+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:14.775726+0000","last_peered":"2024-09-14T07:09:14.775726+0000","last_clean":"2024-09-14T07:09:14.775726+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:14.775726+0000","last_undegraded":"2024-09-14T07:09:14.775726+0000","last_fullsized":"2024-09-14T07:09:14.775726+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"83'128","reported_seq":1213,"reported_epoch":84,"state":"remapped+peering","last_fresh":"2024-09-14T07:09:10.342666+0000","last_change":"2024-09-14T07:09:11.339529+0000","last_active":"2024-09-14T07:09:10.342666+0000","last_peered":"2024-09-14T07:09:08.925280+0000","last_clean":"2024-09-14T07:09:08.925280+0000","last_became_active":"2024-09-14T07:08:49.220855+0000","last_became_peered":"2024-09-14T07:08:49.220855+0000","last_unstale":"2024-09-14T07:09:10.342666+0000","last_undegraded":"2024-09-14T07:09:10.342666+0000","last_fullsized":"2024-09-14T07:09:10.342666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":13242921,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":973,"num_read_kb":2098,"num_write":113,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":128082,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.772406+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:14.772406+0000","last_peered":"2024-09-14T07:09:14.772406+0000","last_clean":"2024-09-14T07:09:14.772406+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:14.772406+0000","last_undegraded":"2024-09-14T07:09:14.772406+0000","last_fullsized":"2024-09-14T07:09:14.772406+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"82'142","reported_seq":1565,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.986596+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:14.986596+0000","last_peered":"2024-09-14T07:09:14.986596+0000","last_clean":"2024-09-14T07:09:14.986596+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:14.986596+0000","last_undegraded":"2024-09-14T07:09:14.986596+0000","last_fullsized":"2024-09-14T07:09:14.986596+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:06.647234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":42916896,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":1301,"num_read_kb":3623,"num_write":125,"num_write_kb":1528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.768842+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:14.768842+0000","last_peered":"2024-09-14T07:09:14.768842+0000","last_clean":"2024-09-14T07:09:14.768842+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:14.768842+0000","last_undegraded":"2024-09-14T07:09:14.768842+0000","last_fullsized":"2024-09-14T07:09:14.768842+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":275,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.772180+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:09:14.772180+0000","last_peered":"2024-09-14T07:09:14.772180+0000","last_clean":"2024-09-14T07:09:14.772180+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:09:14.772180+0000","last_undegraded":"2024-09-14T07:09:14.772180+0000","last_fullsized":"2024-09-14T07:09:14.772180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'195","reported_seq":7022,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.273677+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:15.273677+0000","last_peered":"2024-09-14T07:09:15.273677+0000","last_clean":"2024-09-14T07:09:15.273677+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:15.273677+0000","last_undegraded":"2024-09-14T07:09:15.273677+0000","last_fullsized":"2024-09-14T07:09:15.273677+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:03:14.897298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":120876100,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":3,"num_read":6607,"num_read_kb":57417,"num_write":346,"num_write_kb":24799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"83'357","reported_seq":3728,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-14T07:09:11.338343+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:11.338343+0000","last_peered":"2024-09-14T07:09:11.338343+0000","last_clean":"2024-09-14T07:09:11.338343+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:11.338343+0000","last_undegraded":"2024-09-14T07:09:11.338343+0000","last_fullsized":"2024-09-14T07:09:11.338343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:56:23.911877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":63159631,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":4923,"num_read_kb":24827,"num_write":413,"num_write_kb":6242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'102","reported_seq":3001,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.129861+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:15.129861+0000","last_peered":"2024-09-14T07:09:15.129861+0000","last_clean":"2024-09-14T07:09:15.129861+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:15.129861+0000","last_undegraded":"2024-09-14T07:09:15.129861+0000","last_fullsized":"2024-09-14T07:09:15.129861+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:13:01.735294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":15503888,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":2726,"num_read_kb":8413,"num_write":109,"num_write_kb":1730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"89'582","reported_seq":5035,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.217783+0000","last_change":"2024-09-14T07:09:14.798144+0000","last_active":"2024-09-14T07:09:15.217783+0000","last_peered":"2024-09-14T07:09:15.217783+0000","last_clean":"2024-09-14T07:09:15.217783+0000","last_became_active":"2024-09-14T07:09:14.796081+0000","last_became_peered":"2024-09-14T07:09:14.796081+0000","last_unstale":"2024-09-14T07:09:15.217783+0000","last_undegraded":"2024-09-14T07:09:15.217783+0000","last_fullsized":"2024-09-14T07:09:15.217783+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:12:07.418976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":44270893,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":7199,"num_read_kb":28001,"num_write":868,"num_write_kb":10813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":17139225,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'175","reported_seq":3067,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.261238+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:09:15.261238+0000","last_peered":"2024-09-14T07:09:15.261238+0000","last_clean":"2024-09-14T07:09:15.261238+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:09:15.261238+0000","last_undegraded":"2024-09-14T07:09:15.261238+0000","last_fullsized":"2024-09-14T07:09:15.261238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:02:00.996952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":37245440,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":3927,"num_read_kb":14736,"num_write":131,"num_write_kb":5564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"83'116","reported_seq":2530,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.108126+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:15.108126+0000","last_peered":"2024-09-14T07:09:15.108126+0000","last_clean":"2024-09-14T07:09:15.108126+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:15.108126+0000","last_undegraded":"2024-09-14T07:09:15.108126+0000","last_fullsized":"2024-09-14T07:09:15.108126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:28:10.120936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":21664768,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":2293,"num_read_kb":11564,"num_write":105,"num_write_kb":2238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"83'215","reported_seq":4792,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.138589+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:15.138589+0000","last_peered":"2024-09-14T07:09:15.138589+0000","last_clean":"2024-09-14T07:09:15.138589+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:15.138589+0000","last_undegraded":"2024-09-14T07:09:15.138589+0000","last_fullsized":"2024-09-14T07:09:15.138589+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:24:27.661686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":32686135,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":5500,"num_read_kb":18281,"num_write":169,"num_write_kb":3827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"83'135","reported_seq":1020,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:14.987727+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:14.987727+0000","last_peered":"2024-09-14T07:09:14.987727+0000","last_clean":"2024-09-14T07:09:14.987727+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:14.987727+0000","last_undegraded":"2024-09-14T07:09:14.987727+0000","last_fullsized":"2024-09-14T07:09:14.987727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:40:51.785068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":24228444,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1030,"num_read_kb":5713,"num_write":272,"num_write_kb":13588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"83'111","reported_seq":3904,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.125330+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:09:15.125330+0000","last_peered":"2024-09-14T07:09:15.125330+0000","last_clean":"2024-09-14T07:09:15.125330+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:09:15.125330+0000","last_undegraded":"2024-09-14T07:09:15.125330+0000","last_fullsized":"2024-09-14T07:09:15.125330+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:43:15.525870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":29931008,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":3294,"num_read_kb":7644,"num_write":205,"num_write_kb":12564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'241","reported_seq":5189,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.286522+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:15.286522+0000","last_peered":"2024-09-14T07:09:15.286522+0000","last_clean":"2024-09-14T07:09:15.286522+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:15.286522+0000","last_undegraded":"2024-09-14T07:09:15.286522+0000","last_fullsized":"2024-09-14T07:09:15.286522+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:42:34.044622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":42029711,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":5736,"num_read_kb":10072,"num_write":182,"num_write_kb":6612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"89'108","reported_seq":2316,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.282392+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:09:15.282392+0000","last_peered":"2024-09-14T07:09:15.282392+0000","last_clean":"2024-09-14T07:09:15.282392+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:09:15.282392+0000","last_undegraded":"2024-09-14T07:09:15.282392+0000","last_fullsized":"2024-09-14T07:09:15.282392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:33:27.287506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":46348800,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2069,"num_read_kb":13384,"num_write":95,"num_write_kb":5863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"83'126","reported_seq":2216,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:09:15.110847+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:15.110847+0000","last_peered":"2024-09-14T07:09:15.110847+0000","last_clean":"2024-09-14T07:09:15.110847+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:15.110847+0000","last_undegraded":"2024-09-14T07:09:15.110847+0000","last_fullsized":"2024-09-14T07:09:15.110847+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:27.871664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":64163344,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":3362,"num_read_kb":36116,"num_write":137,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1297416280,"num_objects":597,"num_object_clones":87,"num_object_copies":1194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":597,"num_whiteouts":175,"num_read":118525,"num_read_kb":538331,"num_write":7518,"num_write_kb":208166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":127419815,"num_keys_recovered":276,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26931200,"data_stored":306339063,"data_compressed":8453593,"data_compressed_allocated":22626304,"data_compressed_original":303812096,"omap_allocated":0,"internal_metadata":0},"log_size":6091,"ondisk_log_size":6091,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34160,"kb_used_data":7128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337680,"statfs":{"total":96636764160,"available":96601784320,"internally_reserved":0,"allocated":7299072,"data_stored":74510309,"data_compressed":1785640,"data_compressed_allocated":5246976,"data_compressed_original":73461248,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607577,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42928,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328912,"statfs":{"total":96636764160,"available":96592805888,"internally_reserved":0,"allocated":3502080,"data_stored":19331055,"data_compressed":1034055,"data_compressed_allocated":2027520,"data_compressed_original":18667520,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607576,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45932,"kb_used_data":6416,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325908,"statfs":{"total":96636764160,"available":96589729792,"internally_reserved":0,"allocated":6569984,"data_stored":58402666,"data_compressed":1995573,"data_compressed_allocated":4677632,"data_compressed_original":57482240,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607578,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45700,"kb_used_data":6184,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326140,"statfs":{"total":96636764160,"available":96589967360,"internally_reserved":0,"allocated":6332416,"data_stored":58484347,"data_compressed":1826712,"data_compressed_allocated":4440064,"data_compressed_original":57569792,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607578,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":5264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":5390336,"data_stored":57774857,"data_compressed":1257545,"data_compressed_allocated":3723264,"data_compressed_original":57040384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607578,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46088,"kb_used_data":6576,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325752,"statfs":{"total":96636764160,"available":96589570048,"internally_reserved":0,"allocated":6733824,"data_stored":68585329,"data_compressed":1606220,"data_compressed_allocated":4890624,"data_compressed_original":67615744,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5328896,"data_stored":64057233,"data_compressed":1479842,"data_compressed_allocated":4534272,"data_compressed_original":63512064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4116480,"data_stored":55747736,"data_compressed":1070558,"data_compressed_allocated":3530752,"data_compressed_original":55455232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4206592,"data_stored":43815956,"data_compressed":1502104,"data_compressed_allocated":3506176,"data_compressed_original":43450880,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4710400,"data_stored":51655265,"data_compressed":1693516,"data_compressed_allocated":3993600,"data_compressed_original":51231744,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6307840,"data_stored":74113195,"data_compressed":1785640,"data_compressed_allocated":5246976,"data_compressed_original":73461248,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2260992,"data_stored":16949678,"data_compressed":921933,"data_compressed_allocated":1814528,"data_compressed_original":16700928,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:16.226 INFO:tasks.thrashosds.thrasher:Setting 3.6 to [3, 0, 2, 5] 2024-09-14T07:09:16.226 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.6', '3', '0', '2', '5'] 2024-09-14T07:09:16.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.6 3 0 2 5 2024-09-14T07:09:16.226 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:16.216+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:16.322 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:16.316+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:16.409 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:16.409 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:16.409 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:16.410 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:16.423 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:16.420+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:16.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:16.524 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:16.520+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:16.541 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T07:09:16.599 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:16.599 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:16.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:16.659 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:16.656+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:16.726 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:16.720+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:16.732 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:16.732 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:16.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.050599843000001, 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6596240059999996, 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:09:16.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:09:16.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6595942079999997, 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.742 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:16.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6595619329999991, 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:16.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:09:16.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6595038300000002, 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:09:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6594629279999999, 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6594011200000001, 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:09:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:09:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6593668039999994, 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:09:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6593481329999999, 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:09:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:09:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6593240970000007, 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6593040630000004, 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:09:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6592892280000004, 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:09:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.659274602, 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:09:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6592601009999992, 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6592423459999992, 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:09:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6592274519999997, 2024-09-14T07:09:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:09:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6592133409999992, 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6591914120000002, 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6591701010000008, 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:09:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:09:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.3718192, 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:09:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:09:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:09:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:09:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:09:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:16.815 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:16.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:16.817 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.6 pg_upmap_items mapping to [3->0,2->5] 2024-09-14T07:09:16.817 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:09:16.864 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:16.860+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:16.927 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:16.924+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:16.982 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:16.982 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:16.982 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:16.982 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:16.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:17.039 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:17.036+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:17.129 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:17.124+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:17.166 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:17.166 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:17.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:17.230 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:17.224+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:17.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:17.310 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:17.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.009405121, 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:09:17.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:17.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:09:17.316 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.964414814999998, 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:17.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:17.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:09:17.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:09:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.662482359000002, 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:09:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:09:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.341154625000001, 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:09:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:09:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.904026225999999, 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:09:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.904015568, 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:09:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.904000554, 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:09:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.903989639000001, 2024-09-14T07:09:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.903978129, 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:09:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.903965586, 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:09:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897129584, 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897118989999999, 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:09:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897071209, 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897057741999999, 2024-09-14T07:09:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897046769999999, 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:09:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897036008000001, 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:09:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897019953999999, 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:09:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:09:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.897003382999999, 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.896991607, 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.89698093, 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:17.404 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:17.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:17.406 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:17.400+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:17.506 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:17.500+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:17.550 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:17.550 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:17.550 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:17.566 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:17.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-14T07:09:17.576+0000 7f0e467fc640 0 --2- 172.21.15.5:0/2384440696 >> [v2:172.21.15.5:3301/0,v1:172.21.15.5:6790/0] conn(0x7f0e4809fc30 0x7f0e4809eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-14T07:09:17.611 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:17.608+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:17.616 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:17.642 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:17.664 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:17.693 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:17.699 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:17.699 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:17.707 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:17.704+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:17.714 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:17.715 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:17.715 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:17.737 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:17.737 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:17.808 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:17.804+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:17.896 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:17.897 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:17.909 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:17.904+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:17.912 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:18.010 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:18.004+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:18.079 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:18.079 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:18.093 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:18.113 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:18.108+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:18.214 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:18.208+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:18.256 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:18.257 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:18.271 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:18.315 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:18.312+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:18.416 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:18.412+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:18.441 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:18.442 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:18.456 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:18.517 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:18.512+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:18.586 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:18.587 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:18.602 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:18.618 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:18.612+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:18.719 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:18.716+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:18.795 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:18.795 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:18.812 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:18.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:09:18.820 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:18.816+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:18.921 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:18.916+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:18.930 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:18.930 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:18.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:19.021 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:19.016+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:19.081 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:19.082 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:19.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:19.139 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:19.136+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:19.223 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:19.220+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:19.238 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:09:19.263 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:19.263 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:19.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:19.363 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:19.360+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:19.425 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:19.420+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:19.439 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:19.439 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:19.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:19.454 INFO:teuthology.orchestra.run.smithi145.stdout:2813 punch from 0xb48fa00 to 0xb495000, (0x5600 bytes) 2024-09-14T07:09:19.479 INFO:teuthology.orchestra.run.smithi145.stdout:2816 read 0x48dca00 thru 0x48e8a81 (0xc082 bytes) 2024-09-14T07:09:19.481 INFO:teuthology.orchestra.run.smithi145.stdout:2817 punch from 0x11f4a00 to 0x11f6000, (0x1600 bytes) 2024-09-14T07:09:19.541 INFO:teuthology.orchestra.run.smithi145.stdout:2819 write 0xb6b6c00 thru 0xb6c2d8e (0xc18f bytes) 2024-09-14T07:09:19.542 INFO:teuthology.orchestra.run.smithi145.stdout:2822 read 0x76a800 thru 0x76bf47 (0x1748 bytes) 2024-09-14T07:09:19.543 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:19.540+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:19.618 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:19.618 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:19.627 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:19.624+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:19.627 INFO:teuthology.orchestra.run.smithi145.stdout:2823 trunc from 0xe3de200 to 0x2ee8c00 2024-09-14T07:09:19.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:19.727 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:19.724+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:19.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:19.775 INFO:teuthology.orchestra.run.smithi145.stdout:2824 read 0x2a3e600 thru 0x2a40044 (0x1a45 bytes) 2024-09-14T07:09:19.794 INFO:teuthology.orchestra.run.smithi145.stdout:2825 trunc from 0x2ee8c00 to 0x9d1fe00 2024-09-14T07:09:19.815 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:19.815 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:19.828 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:19.824+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:19.829 INFO:teuthology.orchestra.run.smithi145.stdout:2827 read 0x49ce400 thru 0x49dccfd (0xe8fe bytes) 2024-09-14T07:09:19.831 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:19.834 INFO:teuthology.orchestra.run.smithi145.stdout:2828 read 0x4281c00 thru 0x4290f03 (0xf304 bytes) 2024-09-14T07:09:19.835 INFO:teuthology.orchestra.run.smithi145.stdout:2830 read 0x2a6ee00 thru 0x2a7a9ad (0xbbae bytes) 2024-09-14T07:09:19.837 INFO:teuthology.orchestra.run.smithi145.stdout:2831 read 0x58af200 thru 0x58bcf7d (0xdd7e bytes) 2024-09-14T07:09:19.838 INFO:teuthology.orchestra.run.smithi145.stdout:2832 punch from 0x7129a00 to 0x7139000, (0xf600 bytes) 2024-09-14T07:09:19.838 INFO:teuthology.orchestra.run.smithi145.stdout:2833 read 0x270de00 thru 0x2711468 (0x3669 bytes) 2024-09-14T07:09:19.842 INFO:teuthology.orchestra.run.smithi145.stdout:2835 write 0x71b7800 thru 0x71c4368 (0xcb69 bytes) 2024-09-14T07:09:19.854 INFO:teuthology.orchestra.run.smithi145.stdout:2837 trunc from 0x9d1fe00 to 0x72a8800 2024-09-14T07:09:19.903 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:19.903 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:19.907 INFO:teuthology.orchestra.run.smithi145.stdout:2839 trunc from 0x72a8800 to 0x262a800 2024-09-14T07:09:19.920 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:19.920 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:19.929 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:19.924+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:19.937 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:19.938 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:19.950 INFO:teuthology.orchestra.run.smithi145.stdout:2840 read 0x1cc1a00 thru 0x1cc98cf (0x7ed0 bytes) 2024-09-14T07:09:19.953 INFO:teuthology.orchestra.run.smithi145.stdout:2842 punch from 0x11e4800 to 0x11f4400, (0xfc00 bytes) 2024-09-14T07:09:19.960 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:19.960 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:19.961 INFO:teuthology.orchestra.run.smithi145.stdout:2843 trunc from 0x262a800 to 0x4f1d400 2024-09-14T07:09:19.980 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:19.981 INFO:teuthology.orchestra.run.smithi145.stdout:2844 punch from 0x26f7000 to 0x26fd400, (0x6400 bytes) 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.313629413999998, 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922653577, 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:09:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922623779, 2024-09-14T07:09:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922591504, 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:09:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922533401000001, 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:09:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:09:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:09:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922492499000001, 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:09:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:09:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922430691000001, 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:09:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922396375, 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922377704000001, 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:09:20.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.030 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922353668, 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:09:20.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922333633999999, 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:09:20.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922318798999999, 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:09:20.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922304173000001, 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.043 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922289672, 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:09:20.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922271917, 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:09:20.051 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:09:20.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:20.055 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922257023, 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:09:20.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:09:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922242912, 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:09:20.059 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:09:20.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:09:20.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922220983000001, 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:09:20.063 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:09:20.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:09:20.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.922199672, 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:09:20.067 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:09:20.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6348487709999997, 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.076 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:20.077 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:20.078 INFO:teuthology.orchestra.run.smithi145.stdout:2846 write 0xdd64000 thru 0xdd66bf7 (0x2bf8 bytes) 2024-09-14T07:09:20.078 INFO:teuthology.orchestra.run.smithi145.stdout:2848 punch from 0x8e92800 to 0x8e93c00, (0x1400 bytes) 2024-09-14T07:09:20.078 INFO:teuthology.orchestra.run.smithi145.stdout:2849 read 0xd2c8600 thru 0xd2d5ece (0xd8cf bytes) 2024-09-14T07:09:20.079 INFO:teuthology.orchestra.run.smithi145.stdout:2855 punch from 0x8ce7400 to 0x8cf5000, (0xdc00 bytes) 2024-09-14T07:09:20.079 INFO:teuthology.orchestra.run.smithi145.stdout:2856 trunc from 0xdd66c00 to 0x191c00 2024-09-14T07:09:20.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:20.080 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:20.072+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:20.125 INFO:teuthology.orchestra.run.smithi145.stdout:2857 read 0x55200 thru 0x5ab63 (0x5964 bytes) 2024-09-14T07:09:20.139 INFO:teuthology.orchestra.run.smithi145.stdout:2860 trunc from 0x191c00 to 0x35cca00 2024-09-14T07:09:20.179 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:20.172+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:20.195 INFO:teuthology.orchestra.run.smithi145.stdout:2861 write 0x7322e00 thru 0x7323970 (0xb71 bytes) 2024-09-14T07:09:20.228 INFO:teuthology.orchestra.run.smithi145.stdout:2862 write 0x88c4c00 thru 0x88c984f (0x4c50 bytes) 2024-09-14T07:09:20.230 INFO:teuthology.orchestra.run.smithi145.stdout:2865 read 0x75f1800 thru 0x75f239b (0xb9c bytes) 2024-09-14T07:09:20.246 INFO:teuthology.orchestra.run.smithi145.stdout:2866 trunc from 0x88c9a00 to 0xe5d9400 2024-09-14T07:09:20.255 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:20.256 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:20.270 INFO:teuthology.orchestra.run.smithi145.stdout:2868 punch from 0x9ec8c00 to 0x9ec9200, (0x600 bytes) 2024-09-14T07:09:20.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:20.296 INFO:teuthology.orchestra.run.smithi145.stdout:2869 write 0xea3d600 thru 0xea44367 (0x6d68 bytes) 2024-09-14T07:09:20.298 INFO:teuthology.orchestra.run.smithi145.stdout:2872 write 0xd293400 thru 0xd2a0f5f (0xdb60 bytes) 2024-09-14T07:09:20.298 INFO:teuthology.orchestra.run.smithi145.stdout:2873 read 0x6f39800 thru 0x6f3df29 (0x472a bytes) 2024-09-14T07:09:20.299 INFO:teuthology.orchestra.run.smithi145.stdout:2874 punch from 0xbe16000 to 0xbe1b200, (0x5200 bytes) 2024-09-14T07:09:20.309 INFO:teuthology.orchestra.run.smithi145.stdout:2875 trunc from 0xea44400 to 0xa16b000 2024-09-14T07:09:20.315 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:20.312+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:20.349 INFO:teuthology.orchestra.run.smithi145.stdout:2876 read 0x8851000 thru 0x88563d1 (0x53d2 bytes) 2024-09-14T07:09:20.380 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:20.376+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:20.384 INFO:teuthology.orchestra.run.smithi145.stdout:2877 write 0xe95c000 thru 0xe96845a (0xc45b bytes) 2024-09-14T07:09:20.386 INFO:teuthology.orchestra.run.smithi145.stdout:2878 punch from 0x47e2800 to 0x47ea400, (0x7c00 bytes) 2024-09-14T07:09:20.387 INFO:teuthology.orchestra.run.smithi145.stdout:2881 read 0x7f07200 thru 0x7f07727 (0x528 bytes) 2024-09-14T07:09:20.388 INFO:teuthology.orchestra.run.smithi145.stdout:2884 write 0x97c5e00 thru 0x97c84c1 (0x26c2 bytes) 2024-09-14T07:09:20.389 INFO:teuthology.orchestra.run.smithi145.stdout:2886 write 0x846f400 thru 0x846fbc8 (0x7c9 bytes) 2024-09-14T07:09:20.390 INFO:teuthology.orchestra.run.smithi145.stdout:2887 write 0x3324400 thru 0x332c538 (0x8139 bytes) 2024-09-14T07:09:20.392 INFO:teuthology.orchestra.run.smithi145.stdout:2888 write 0x1aa1000 thru 0x1aad274 (0xc275 bytes) 2024-09-14T07:09:20.395 INFO:teuthology.orchestra.run.smithi145.stdout:2889 write 0x6addc00 thru 0x6ae5810 (0x7c11 bytes) 2024-09-14T07:09:20.417 INFO:teuthology.orchestra.run.smithi145.stdout:2890 trunc from 0xe968600 to 0x29cda00 2024-09-14T07:09:20.445 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:20.445 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:20.445 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:20.445 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:20.445 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:20.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:20.496 INFO:teuthology.orchestra.run.smithi145.stdout:2892 write 0x6b03600 thru 0x6b10f8f (0xd990 bytes) 2024-09-14T07:09:20.503 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:20.500+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:20.537 INFO:teuthology.orchestra.run.smithi145.stdout:2894 write 0xcc66c00 thru 0xcc6c24d (0x564e bytes) 2024-09-14T07:09:20.539 INFO:teuthology.orchestra.run.smithi145.stdout:2897 read 0x674dc00 thru 0x6754187 (0x6588 bytes) 2024-09-14T07:09:20.539 INFO:teuthology.orchestra.run.smithi145.stdout:2898 write 0x1950c00 thru 0x195e4bc (0xd8bd bytes) 2024-09-14T07:09:20.542 INFO:teuthology.orchestra.run.smithi145.stdout:2899 punch from 0x2fde00 to 0x303e00, (0x6000 bytes) 2024-09-14T07:09:20.546 INFO:teuthology.orchestra.run.smithi145.stdout:2900 trunc from 0xcc6c400 to 0xbcf2a00 2024-09-14T07:09:20.582 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:20.576+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:20.592 INFO:teuthology.orchestra.run.smithi145.stdout:2905 trunc from 0xbcf2a00 to 0xc6ff200 2024-09-14T07:09:20.623 INFO:teuthology.orchestra.run.smithi145.stdout:2906 trunc from 0xc6ff200 to 0x9881e00 2024-09-14T07:09:20.629 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.328399204999997, 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.635 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.283408899000001, 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:09:20.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:09:20.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.981476442999998, 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:09:20.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:09:20.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.646 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.660148708999998, 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:09:20.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:09:20.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.652 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.223020310000001, 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.656 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.223009652, 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:20.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:09:20.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.222994637999999, 2024-09-14T07:09:20.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:09:20.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.222983723, 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.666 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:20.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.222972213, 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.674 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.22295967, 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:20.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.216123668, 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:09:20.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.216113074000001, 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:20.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.687 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.216065293, 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:09:20.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.216051825999999, 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:20.696 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.216040853999999, 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:09:20.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.216030092, 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:20.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:20.705 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.216014038000001, 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:20.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.215997466999999, 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.710 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.215985691, 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:09:20.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.215975014, 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:20.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:09:20.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:20.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:09:20.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:09:20.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.723 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:20.723 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:20.723 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:20.724 INFO:teuthology.orchestra.run.smithi145.stdout:2907 punch from 0x785a000 to 0x7862400, (0x8400 bytes) 2024-09-14T07:09:20.724 INFO:teuthology.orchestra.run.smithi145.stdout:2910 write 0x6766600 thru 0x676b6ac (0x50ad bytes) 2024-09-14T07:09:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:2911 write 0x9406a00 thru 0x940b0d6 (0x46d7 bytes) 2024-09-14T07:09:20.725 INFO:teuthology.orchestra.run.smithi145.stdout:2912 clone 21 order 18 su 65536 sc 3 2024-09-14T07:09:20.725 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:20.726 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:20.720+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:20.825 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:20.820+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:20.856 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:20.857 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:20.870 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:20.956 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:20.952+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:21.026 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:21.020+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:21.035 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:21.035 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:21.035 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:21.035 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:21.035 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:21.049 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:21.127 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:21.124+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:21.217 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 108.91408817999999, 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:21.218 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:21.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:21.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:21.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:21.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:21.224 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.020094995999997, 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:21.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:21.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:21.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:21.232 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.019975117000001, 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:21.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:21.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.002749997000002, 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.240 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.245 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.33015561, 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:09:21.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:09:21.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5561137439999992, 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5560815160000008, 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:09:21.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:09:21.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.259 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.556059501, 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:09:21.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.263 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5560385659999998, 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:09:21.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5560175560000005, 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.268 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:09:21.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:09:21.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5559838559999992, 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.272 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:09:21.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:09:21.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5559632130000001, 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.277 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:09:21.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.280 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5557179960000003, 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:09:21.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:09:21.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:09:21.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5556862769999995, 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:09:21.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:09:21.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:09:21.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5556645180000004, 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:09:21.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:09:21.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.555632439, 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.293 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.297 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5544612329999996, 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:09:21.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:09:21.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5544382700000003, 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:21.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:09:21.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5544169799999992, 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:09:21.306 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:21.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:09:21.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5543967569999992, 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.311 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:09:21.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.315 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:21.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:09:21.317 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:21.317 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:21.312+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:21.403 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone20 from 0x9881e00 (overlap 0x191c00) to 0x0 2024-09-14T07:09:21.417 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:21.412+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:21.498 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:21.498 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:21.498 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:21.498 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:21.512 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:21.563 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:21.560+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:21.619 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:21.612+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:21.628 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:21.628 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:21.628 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:21.628 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:21.628 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:21.642 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:21.731 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:21.728+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:21.819 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:21.819 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:21.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 64.520370528000001, 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:21.820 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:21.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:21.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.660894941000002, 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:21.826 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:21.830 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.660792655999998, 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:21.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.961740814999999, 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:21.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476819623000001, 2024-09-14T07:09:21.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:09:21.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476784403, 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:21.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476729254, 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:09:21.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:21.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476695185000001, 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:09:21.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476651839000001, 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:21.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476618316, 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.861 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476571066, 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:09:21.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476456818000001, 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:09:21.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:09:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476347123, 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.47628098, 2024-09-14T07:09:21.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:21.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.880 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476164475999999, 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476098820000001, 2024-09-14T07:09:21.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.476039500000001, 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.888 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:21.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.475898632, 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:09:21.895 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.475483525, 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.896 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 15.475415785999999, 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.900 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:09:21.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:21.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:21.903 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:21.904 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:21.904 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:21.816+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:21.905 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:21.905 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:09:21.905 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:21.905 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T07:09:21.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:09:21.906 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T07:09:21.948 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:21.944+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:22.021 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:22.016+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:22.081 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:22.084 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:22.084 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:22.084 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:22.084 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:22.098 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:22.127 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:22.124+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:22.223 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:22.216+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:22.226 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:22.226 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:22.226 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:22.226 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:22.226 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:22.239 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:22.239 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:22.241 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:22.257 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:22.257 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:22.257 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 93, 2024-09-14T07:09:22.257 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:09:22.257 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:09:21.248166+0000", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:09:10.322879+0000", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5799936, 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:09:22.258 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "luminous", 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:09:22.259 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:09:22.260 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:09:22.261 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:09:22.262 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:09:22.263 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 6.059999942779541, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 6.059999942779541, 2024-09-14T07:09:22.264 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.33000001311302185, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:09:22.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:09:22.266 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:09:22.267 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:09:22.268 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:09:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.270 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 2.25, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2.25, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:09:22.271 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:09:22.272 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:09:22.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "93", 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 23, 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 93, 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:09:22.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.275 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:09:22.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.690000057220459, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.690000057220459, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.690000057220459, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.690000057220459, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.88294988870620728, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.87999999523162842, 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:22.277 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.297698974609375, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:22.278 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:22.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:22.280 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:22.281 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:09:22.282 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:22.283 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:09:22.284 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.285 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:22.288 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:22.293 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:22.294 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:22.295 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.297 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:22.298 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:22.299 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.302 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:09:22.303 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:09:22.304 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 88, 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:09:22.305 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.306 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.308 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:22.309 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:22.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:22.311 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:09:22.312 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [ 2024-09-14T07:09:22.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "pgid": "3.13", 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mappings": [ 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "from": 3, 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "to": 1 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:09:22.314 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:09:22.315 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:09:22.316 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:09:22.316 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:09:22.316 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:09:22.316 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:09:22.316 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:22.316 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:22.316 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:22.317 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:22.317 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:09:22.317 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:22.317 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:22.319 INFO:tasks.thrashosds.thrasher:j is {'epoch': 93, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:09:21.248166+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:09:10.322879+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.8829498887062073, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.297698974609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 88, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.13', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:09:22.319 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:22.320 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:22.322 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:22.323 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:22.320+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:22.341 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:22.375 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:22.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:22.424 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:22.420+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:22.524 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:22.520+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:22.525 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:22.525 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:22.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:22.627 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:22.624+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:22.646 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:22.646 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:22.665 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-14T07:09:22.258680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1446157030,"num_objects":609,"num_object_clones":87,"num_object_copies":1218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":609,"num_whiteouts":175,"num_read":136324,"num_read_kb":585361,"num_write":7825,"num_write_kb":217461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1004,"num_bytes_recovered":128743440,"num_keys_recovered":288,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6379,"ondisk_log_size":6379,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":249068,"kb_used_data":32820,"kb_used_omap":9,"kb_used_meta":216118,"kb_avail":565981972,"statfs":{"total":579820584960,"available":579565539328,"internally_reserved":0,"allocated":33607680,"data_stored":290360493,"data_compressed":9096639,"data_compressed_allocated":22683648,"data_compressed_original":285227520,"omap_allocated":9568,"internal_metadata":221305504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":148150363,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":17733,"num_read_kb":46975,"num_write":204,"num_write_kb":7665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":142889,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.470844"},"pg_stats":[{"pgid":"3.1f","version":"82'109","reported_seq":4906,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.285248+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:09:19.285248+0000","last_peered":"2024-09-14T07:09:19.285248+0000","last_clean":"2024-09-14T07:09:19.285248+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:09:19.285248+0000","last_undegraded":"2024-09-14T07:09:19.285248+0000","last_fullsized":"2024-09-14T07:09:19.285248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:36.618117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":68849680,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":4381,"num_read_kb":43054,"num_write":184,"num_write_kb":9642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"82'325","reported_seq":4947,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.262088+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:19.262088+0000","last_peered":"2024-09-14T07:09:19.262088+0000","last_clean":"2024-09-14T07:09:19.262088+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:19.262088+0000","last_undegraded":"2024-09-14T07:09:19.262088+0000","last_fullsized":"2024-09-14T07:09:19.262088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:14:06.379933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":58980591,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":5608,"num_read_kb":36664,"num_write":550,"num_write_kb":12260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"83'300","reported_seq":4411,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.354993+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:19.354993+0000","last_peered":"2024-09-14T07:09:19.354993+0000","last_clean":"2024-09-14T07:09:19.354993+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:19.354993+0000","last_undegraded":"2024-09-14T07:09:19.354993+0000","last_fullsized":"2024-09-14T07:09:19.354993+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:56:47.170417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":8611877,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":5613,"num_read_kb":13831,"num_write":232,"num_write_kb":2184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"90'106","reported_seq":1671,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.263738+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:19.263738+0000","last_peered":"2024-09-14T07:09:19.263738+0000","last_clean":"2024-09-14T07:09:19.263738+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:19.263738+0000","last_undegraded":"2024-09-14T07:09:19.263738+0000","last_fullsized":"2024-09-14T07:09:19.263738+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:18:24.677020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":22163508,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":1432,"num_read_kb":9029,"num_write":115,"num_write_kb":3376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"89'340","reported_seq":7013,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.732044+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:09:19.732044+0000","last_peered":"2024-09-14T07:09:19.732044+0000","last_clean":"2024-09-14T07:09:19.732044+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:09:19.732044+0000","last_undegraded":"2024-09-14T07:09:19.732044+0000","last_fullsized":"2024-09-14T07:09:19.732044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:41.170936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":101528251,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":7458,"num_read_kb":18344,"num_write":463,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"90'152","reported_seq":9445,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T07:09:17.071516+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:17.071516+0000","last_peered":"2024-09-14T07:09:17.071516+0000","last_clean":"2024-09-14T07:09:17.071516+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:17.071516+0000","last_undegraded":"2024-09-14T07:09:17.071516+0000","last_fullsized":"2024-09-14T07:09:17.071516+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:10:44.357878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":69664941,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":10141,"num_read_kb":20671,"num_write":188,"num_write_kb":9345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"89'120","reported_seq":1671,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.726938+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:19.726938+0000","last_peered":"2024-09-14T07:09:19.726938+0000","last_clean":"2024-09-14T07:09:19.726938+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:19.726938+0000","last_undegraded":"2024-09-14T07:09:19.726938+0000","last_fullsized":"2024-09-14T07:09:19.726938+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:38:16.725742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":22634603,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1509,"num_read_kb":10702,"num_write":137,"num_write_kb":5281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"90'140","reported_seq":2043,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.729919+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:09:19.729919+0000","last_peered":"2024-09-14T07:09:19.729919+0000","last_clean":"2024-09-14T07:09:19.729919+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:09:19.729919+0000","last_undegraded":"2024-09-14T07:09:19.729919+0000","last_fullsized":"2024-09-14T07:09:19.729919+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:46:23.278053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":75275328,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":1734,"num_read_kb":5173,"num_write":259,"num_write_kb":13321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"92'83","reported_seq":1358,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.932510+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:09:19.932510+0000","last_peered":"2024-09-14T07:09:19.932510+0000","last_clean":"2024-09-14T07:09:19.932510+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:09:19.932510+0000","last_undegraded":"2024-09-14T07:09:19.932510+0000","last_fullsized":"2024-09-14T07:09:19.932510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:06.338267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":23439910,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":982,"num_read_kb":8229,"num_write":93,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"92'194","reported_seq":6474,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:20.283548+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:20.283548+0000","last_peered":"2024-09-14T07:09:20.283548+0000","last_clean":"2024-09-14T07:09:20.283548+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:20.283548+0000","last_undegraded":"2024-09-14T07:09:20.283548+0000","last_fullsized":"2024-09-14T07:09:20.283548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:54:29.860166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":47148070,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":6624,"num_read_kb":30925,"num_write":179,"num_write_kb":3817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"83'132","reported_seq":3055,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.354302+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:09:19.354302+0000","last_peered":"2024-09-14T07:09:19.354302+0000","last_clean":"2024-09-14T07:09:19.354302+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:09:19.354302+0000","last_undegraded":"2024-09-14T07:09:19.354302+0000","last_fullsized":"2024-09-14T07:09:19.354302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:22:28.811254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":63338545,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":2924,"num_read_kb":35355,"num_write":218,"num_write_kb":12812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"83'156","reported_seq":4628,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.334436+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:19.334436+0000","last_peered":"2024-09-14T07:09:19.334436+0000","last_clean":"2024-09-14T07:09:19.334436+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:19.334436+0000","last_undegraded":"2024-09-14T07:09:19.334436+0000","last_fullsized":"2024-09-14T07:09:19.334436+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:11:22.342035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":53785104,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":10,"num_read":4713,"num_read_kb":18808,"num_write":126,"num_write_kb":3307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.244314+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:19.244314+0000","last_peered":"2024-09-14T07:09:19.244314+0000","last_clean":"2024-09-14T07:09:19.244314+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:19.244314+0000","last_undegraded":"2024-09-14T07:09:19.244314+0000","last_fullsized":"2024-09-14T07:09:19.244314+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"83'306","reported_seq":5217,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.353432+0000","last_change":"2024-09-14T07:09:14.793462+0000","last_active":"2024-09-14T07:09:19.353432+0000","last_peered":"2024-09-14T07:09:19.353432+0000","last_clean":"2024-09-14T07:09:19.353432+0000","last_became_active":"2024-09-14T07:09:14.787124+0000","last_became_peered":"2024-09-14T07:09:14.787124+0000","last_unstale":"2024-09-14T07:09:19.353432+0000","last_undegraded":"2024-09-14T07:09:19.353432+0000","last_fullsized":"2024-09-14T07:09:19.353432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:40:32.065244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":14473449,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":7022,"num_read_kb":17858,"num_write":327,"num_write_kb":2081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1461720,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.242412+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:19.242412+0000","last_peered":"2024-09-14T07:09:19.242412+0000","last_clean":"2024-09-14T07:09:19.242412+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:19.242412+0000","last_undegraded":"2024-09-14T07:09:19.242412+0000","last_fullsized":"2024-09-14T07:09:19.242412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"80'173","reported_seq":4161,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.262353+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:19.262353+0000","last_peered":"2024-09-14T07:09:19.262353+0000","last_clean":"2024-09-14T07:09:19.262353+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:19.262353+0000","last_undegraded":"2024-09-14T07:09:19.262353+0000","last_fullsized":"2024-09-14T07:09:19.262353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:41.195943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":33835038,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":4560,"num_read_kb":13680,"num_write":209,"num_write_kb":2086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.243777+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:09:19.243777+0000","last_peered":"2024-09-14T07:09:19.243777+0000","last_clean":"2024-09-14T07:09:19.243777+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:09:19.243777+0000","last_undegraded":"2024-09-14T07:09:19.243777+0000","last_fullsized":"2024-09-14T07:09:19.243777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"82'425","reported_seq":2095,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.242193+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:19.242193+0000","last_peered":"2024-09-14T07:09:19.242193+0000","last_clean":"2024-09-14T07:09:19.242193+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:19.242193+0000","last_undegraded":"2024-09-14T07:09:19.242193+0000","last_fullsized":"2024-09-14T07:09:19.242193+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:36:50.351230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":393271,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":3869,"num_read_kb":3707,"num_write":631,"num_write_kb":1502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.242298+0000","last_change":"2024-09-14T07:09:11.349379+0000","last_active":"2024-09-14T07:09:19.242298+0000","last_peered":"2024-09-14T07:09:19.242298+0000","last_clean":"2024-09-14T07:09:19.242298+0000","last_became_active":"2024-09-14T07:09:11.348941+0000","last_became_peered":"2024-09-14T07:09:11.348941+0000","last_unstale":"2024-09-14T07:09:19.242298+0000","last_undegraded":"2024-09-14T07:09:19.242298+0000","last_fullsized":"2024-09-14T07:09:19.242298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:39.931750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"90'79","reported_seq":2044,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.354145+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:19.354145+0000","last_peered":"2024-09-14T07:09:19.354145+0000","last_clean":"2024-09-14T07:09:19.354145+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:19.354145+0000","last_undegraded":"2024-09-14T07:09:19.354145+0000","last_fullsized":"2024-09-14T07:09:19.354145+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:57:44.669422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":8532038,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":1845,"num_read_kb":14310,"num_write":104,"num_write_kb":3508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'65","reported_seq":4806,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T07:09:16.821200+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:09:16.821200+0000","last_peered":"2024-09-14T07:09:16.821200+0000","last_clean":"2024-09-14T07:09:16.821200+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:09:16.821200+0000","last_undegraded":"2024-09-14T07:09:16.821200+0000","last_fullsized":"2024-09-14T07:09:16.821200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:38:26.434710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":45023232,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":4223,"num_read_kb":15485,"num_write":141,"num_write_kb":10548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":168,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.263553+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:19.263553+0000","last_peered":"2024-09-14T07:09:19.263553+0000","last_clean":"2024-09-14T07:09:19.263553+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:19.263553+0000","last_undegraded":"2024-09-14T07:09:19.263553+0000","last_fullsized":"2024-09-14T07:09:19.263553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"92'234","reported_seq":2721,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.729716+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:19.729716+0000","last_peered":"2024-09-14T07:09:19.729716+0000","last_clean":"2024-09-14T07:09:19.729716+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:19.729716+0000","last_undegraded":"2024-09-14T07:09:19.729716+0000","last_fullsized":"2024-09-14T07:09:19.729716+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:58:08.641806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":40164551,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":4587,"num_read_kb":7154,"num_write":155,"num_write_kb":1876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":143,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.262068+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:19.262068+0000","last_peered":"2024-09-14T07:09:19.262068+0000","last_clean":"2024-09-14T07:09:19.262068+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:19.262068+0000","last_undegraded":"2024-09-14T07:09:19.262068+0000","last_fullsized":"2024-09-14T07:09:19.262068+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"91'133","reported_seq":1330,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T07:09:16.820397+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:09:16.820397+0000","last_peered":"2024-09-14T07:09:16.820397+0000","last_clean":"2024-09-14T07:09:16.820397+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:09:16.820397+0000","last_undegraded":"2024-09-14T07:09:16.820397+0000","last_fullsized":"2024-09-14T07:09:16.820397+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:05.416480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":13242921,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":8,"num_read":1042,"num_read_kb":2157,"num_write":113,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.242853+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:19.242853+0000","last_peered":"2024-09-14T07:09:19.242853+0000","last_clean":"2024-09-14T07:09:19.242853+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:19.242853+0000","last_undegraded":"2024-09-14T07:09:19.242853+0000","last_fullsized":"2024-09-14T07:09:19.242853+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"82'142","reported_seq":1594,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.242410+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:19.242410+0000","last_peered":"2024-09-14T07:09:19.242410+0000","last_clean":"2024-09-14T07:09:19.242410+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:19.242410+0000","last_undegraded":"2024-09-14T07:09:19.242410+0000","last_fullsized":"2024-09-14T07:09:19.242410+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:03.600358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":42916896,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":1323,"num_read_kb":3815,"num_write":125,"num_write_kb":1528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":187,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.263786+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:19.263786+0000","last_peered":"2024-09-14T07:09:19.263786+0000","last_clean":"2024-09-14T07:09:19.263786+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:19.263786+0000","last_undegraded":"2024-09-14T07:09:19.263786+0000","last_fullsized":"2024-09-14T07:09:19.263786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":281,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.243144+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:09:19.243144+0000","last_peered":"2024-09-14T07:09:19.243144+0000","last_clean":"2024-09-14T07:09:19.243144+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:09:19.243144+0000","last_undegraded":"2024-09-14T07:09:19.243144+0000","last_fullsized":"2024-09-14T07:09:19.243144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'207","reported_seq":7143,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.733040+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:19.733040+0000","last_peered":"2024-09-14T07:09:19.733040+0000","last_clean":"2024-09-14T07:09:19.733040+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:19.733040+0000","last_undegraded":"2024-09-14T07:09:19.733040+0000","last_fullsized":"2024-09-14T07:09:19.733040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:58.370301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":139599940,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":3,"num_read":6717,"num_read_kb":57616,"num_write":353,"num_write_kb":24923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"89'369","reported_seq":4020,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-14T07:09:17.071570+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:17.071570+0000","last_peered":"2024-09-14T07:09:17.071570+0000","last_clean":"2024-09-14T07:09:17.071570+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:17.071570+0000","last_undegraded":"2024-09-14T07:09:17.071570+0000","last_fullsized":"2024-09-14T07:09:17.071570+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:59:22.724077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":77184335,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":5199,"num_read_kb":27274,"num_write":426,"num_write_kb":8859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"92'110","reported_seq":4069,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:20.284575+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:20.284575+0000","last_peered":"2024-09-14T07:09:20.284575+0000","last_clean":"2024-09-14T07:09:20.284575+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:20.284575+0000","last_undegraded":"2024-09-14T07:09:20.284575+0000","last_fullsized":"2024-09-14T07:09:20.284575+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:51:18.342117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":28912656,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":3781,"num_read_kb":10483,"num_write":116,"num_write_kb":1867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"89'589","reported_seq":5163,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.728642+0000","last_change":"2024-09-14T07:09:14.798144+0000","last_active":"2024-09-14T07:09:19.728642+0000","last_peered":"2024-09-14T07:09:19.728642+0000","last_clean":"2024-09-14T07:09:19.728642+0000","last_became_active":"2024-09-14T07:09:14.796081+0000","last_became_peered":"2024-09-14T07:09:14.796081+0000","last_unstale":"2024-09-14T07:09:19.728642+0000","last_undegraded":"2024-09-14T07:09:19.728642+0000","last_fullsized":"2024-09-14T07:09:19.728642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:12:53.528962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":56853805,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":7316,"num_read_kb":28401,"num_write":874,"num_write_kb":10912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":17139225,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'181","reported_seq":3162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.727146+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:09:19.727146+0000","last_peered":"2024-09-14T07:09:19.727146+0000","last_clean":"2024-09-14T07:09:19.727146+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:09:19.727146+0000","last_undegraded":"2024-09-14T07:09:19.727146+0000","last_fullsized":"2024-09-14T07:09:19.727146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:17:35.010474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":40893440,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":4012,"num_read_kb":16659,"num_write":135,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"83'116","reported_seq":2598,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.354082+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:19.354082+0000","last_peered":"2024-09-14T07:09:19.354082+0000","last_clean":"2024-09-14T07:09:19.354082+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:19.354082+0000","last_undegraded":"2024-09-14T07:09:19.354082+0000","last_fullsized":"2024-09-14T07:09:19.354082+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:26:20.727615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":21664768,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":2354,"num_read_kb":12040,"num_write":105,"num_write_kb":2238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"83'215","reported_seq":5877,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.329361+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:19.329361+0000","last_peered":"2024-09-14T07:09:19.329361+0000","last_clean":"2024-09-14T07:09:19.329361+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:19.329361+0000","last_undegraded":"2024-09-14T07:09:19.329361+0000","last_fullsized":"2024-09-14T07:09:19.329361+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:30.739958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":32686135,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":6578,"num_read_kb":20675,"num_write":169,"num_write_kb":3827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"92'172","reported_seq":1101,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:20.259351+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:20.259351+0000","last_peered":"2024-09-14T07:09:20.259351+0000","last_clean":"2024-09-14T07:09:20.259351+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:20.259351+0000","last_undegraded":"2024-09-14T07:09:20.259351+0000","last_fullsized":"2024-09-14T07:09:20.259351+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:50:50.077428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":24228481,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":1221,"num_read_kb":5955,"num_write":336,"num_write_kb":15350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"92'115","reported_seq":4812,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:20.279218+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:09:20.279218+0000","last_peered":"2024-09-14T07:09:20.279218+0000","last_clean":"2024-09-14T07:09:20.279218+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:09:20.279218+0000","last_undegraded":"2024-09-14T07:09:20.279218+0000","last_fullsized":"2024-09-14T07:09:20.279218+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:13:40.977731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":31576576,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":4195,"num_read_kb":10851,"num_write":219,"num_write_kb":14535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'254","reported_seq":6508,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.731201+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:19.731201+0000","last_peered":"2024-09-14T07:09:19.731201+0000","last_clean":"2024-09-14T07:09:19.731201+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:19.731201+0000","last_undegraded":"2024-09-14T07:09:19.731201+0000","last_fullsized":"2024-09-14T07:09:19.731201+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:54:58.089519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":59202191,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":7043,"num_read_kb":14983,"num_write":193,"num_write_kb":6749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"90'136","reported_seq":2690,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.730652+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:09:19.730652+0000","last_peered":"2024-09-14T07:09:19.730652+0000","last_clean":"2024-09-14T07:09:19.730652+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:09:19.730652+0000","last_undegraded":"2024-09-14T07:09:19.730652+0000","last_fullsized":"2024-09-14T07:09:19.730652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:59:25.169960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":54599168,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2795,"num_read_kb":14596,"num_write":110,"num_write_kb":5966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"83'126","reported_seq":2318,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-14T07:09:19.356011+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:19.356011+0000","last_peered":"2024-09-14T07:09:19.356011+0000","last_clean":"2024-09-14T07:09:19.356011+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:19.356011+0000","last_undegraded":"2024-09-14T07:09:19.356011+0000","last_fullsized":"2024-09-14T07:09:19.356011+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:38:26.633740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":64163344,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":3457,"num_read_kb":36822,"num_write":137,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1445566643,"num_objects":605,"num_object_clones":87,"num_object_copies":1210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":605,"num_whiteouts":175,"num_read":136258,"num_read_kb":585306,"num_write":7722,"num_write_kb":215831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":127562704,"num_keys_recovered":288,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26464256,"data_stored":282638182,"data_compressed":8818696,"data_compressed_allocated":22102016,"data_compressed_original":280160768,"omap_allocated":0,"internal_metadata":0},"log_size":6304,"ondisk_log_size":6304,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":5772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":5910528,"data_stored":48425726,"data_compressed":1571563,"data_compressed_allocated":4022272,"data_compressed_original":47531520,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607578,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43076,"kb_used_data":3568,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328764,"statfs":{"total":96636764160,"available":96592654336,"internally_reserved":0,"allocated":3653632,"data_stored":19122991,"data_compressed":1079887,"data_compressed_allocated":2064384,"data_compressed_original":18405376,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607577,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49680,"kb_used_data":6012,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94322160,"statfs":{"total":96636764160,"available":96585891840,"internally_reserved":0,"allocated":6156288,"data_stored":54438383,"data_compressed":1854913,"data_compressed_allocated":4288512,"data_compressed_original":53539328,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607579,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44824,"kb_used_data":5316,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94327016,"statfs":{"total":96636764160,"available":96590864384,"internally_reserved":0,"allocated":5443584,"data_stored":44409197,"data_compressed":1591412,"data_compressed_allocated":3604480,"data_compressed_original":43581952,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607579,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":5184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":5308416,"data_stored":56283674,"data_compressed":1138246,"data_compressed_allocated":3604480,"data_compressed_original":55543808,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607579,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46476,"kb_used_data":6968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325364,"statfs":{"total":96636764160,"available":96589172736,"internally_reserved":0,"allocated":7135232,"data_stored":67680522,"data_compressed":1860618,"data_compressed_allocated":5099520,"data_compressed_original":66625536,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5767168,"data_stored":65289121,"data_compressed":1752019,"data_compressed_allocated":4870144,"data_compressed_original":64692224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4235264,"data_stored":55852222,"data_compressed":1138246,"data_compressed_allocated":3604480,"data_compressed_original":55543808,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4329472,"data_stored":43917860,"data_compressed":1572453,"data_compressed_allocated":3579904,"data_compressed_original":43532800,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4808704,"data_stored":51750022,"data_compressed":1734338,"data_compressed_allocated":4046848,"data_compressed_original":51307008,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4857856,"data_stored":48002698,"data_compressed":1571563,"data_compressed_allocated":4022272,"data_compressed_original":47531520,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2465792,"data_stored":17826259,"data_compressed":1050077,"data_compressed_allocated":1978368,"data_compressed_original":17553408,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:22.667 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [0, 1, 2, 5] 2024-09-14T07:09:22.667 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '0', '1', '2', '5'] 2024-09-14T07:09:22.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 0 1 2 5 2024-09-14T07:09:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:22.713 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:22.725 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:22.720+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:22.727 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:22.826 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:22.820+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:22.896 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:22.897 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:22.912 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:22.928 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:22.924+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:23.028 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:23.024+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:23.085 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:23.085 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:23.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:23.129 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:23.124+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:23.230 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:23.224+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:23.270 INFO:teuthology.orchestra.run.smithi005.stderr:set 2.0 pg_upmap_items mapping to [0->1,2->5] 2024-09-14T07:09:23.273 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:23.273 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:23.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:23.331 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:23.328+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:23.387 INFO:teuthology.orchestra.run.smithi145.stdout:2913 write 0xa615c00 thru 0xa6205ce (0xa9cf bytes) 2024-09-14T07:09:23.388 INFO:teuthology.orchestra.run.smithi145.stdout:2914 punch from 0x2120c00 to 0x2122600, (0x1a00 bytes) 2024-09-14T07:09:23.389 INFO:teuthology.orchestra.run.smithi145.stdout:2915 punch from 0x3b1a00 to 0x3b5600, (0x3c00 bytes) 2024-09-14T07:09:23.428 INFO:teuthology.orchestra.run.smithi145.stdout:2916 write 0xb448a00 thru 0xb451463 (0x8a64 bytes) 2024-09-14T07:09:23.432 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:23.428+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:23.448 INFO:teuthology.orchestra.run.smithi145.stdout:2918 trunc from 0xb451600 to 0xdcf5600 2024-09-14T07:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:23.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:23.480 INFO:teuthology.orchestra.run.smithi145.stdout:2919 read 0x1fa1c00 thru 0x1fa24d2 (0x8d3 bytes) 2024-09-14T07:09:23.482 INFO:teuthology.orchestra.run.smithi145.stdout:2921 punch from 0xb9f7a00 to 0xba05c00, (0xe200 bytes) 2024-09-14T07:09:23.530 INFO:teuthology.orchestra.run.smithi145.stdout:2923 trunc from 0xdcf5600 to 0x2c84800 2024-09-14T07:09:23.533 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:23.528+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:23.622 INFO:teuthology.orchestra.run.smithi145.stdout:2925 write 0x9813600 thru 0x9814ce5 (0x16e6 bytes) 2024-09-14T07:09:23.634 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:23.628+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:23.637 INFO:teuthology.orchestra.run.smithi145.stdout:2927 trunc from 0x9814e00 to 0x2d8d200 2024-09-14T07:09:23.650 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:23.651 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:23.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:23.701 INFO:teuthology.orchestra.run.smithi145.stdout:2928 trunc from 0x2d8d200 to 0x16bc800 2024-09-14T07:09:23.745 INFO:teuthology.orchestra.run.smithi145.stdout:2930 read 0x13e0e00 thru 0x13ee8eb (0xdaec bytes) 2024-09-14T07:09:23.747 INFO:teuthology.orchestra.run.smithi145.stdout:2931 read 0x64cc00 thru 0x64e603 (0x1a04 bytes) 2024-09-14T07:09:23.747 INFO:teuthology.orchestra.run.smithi145.stdout:2933 punch from 0xa3200 to 0xafc00, (0xca00 bytes) 2024-09-14T07:09:23.747 INFO:teuthology.orchestra.run.smithi145.stdout:2934 punch from 0x1510800 to 0x1514400, (0x3c00 bytes) 2024-09-14T07:09:23.751 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:23.748+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:23.795 INFO:teuthology.orchestra.run.smithi145.stdout:2935 write 0xe44f200 thru 0xe45e923 (0xf724 bytes) 2024-09-14T07:09:23.816 INFO:teuthology.orchestra.run.smithi145.stdout:2936 trunc from 0xe45ea00 to 0x6618000 2024-09-14T07:09:23.836 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:23.832+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:23.838 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:23.838 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:23.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:23.867 INFO:teuthology.orchestra.run.smithi145.stdout:2938 read 0x2da3c00 thru 0x2db1c3e (0xe03f bytes) 2024-09-14T07:09:23.902 INFO:teuthology.orchestra.run.smithi145.stdout:2941 write 0x9b1c200 thru 0x9b22705 (0x6506 bytes) 2024-09-14T07:09:23.904 INFO:teuthology.orchestra.run.smithi145.stdout:2944 write 0x814e400 thru 0x814e507 (0x108 bytes) 2024-09-14T07:09:23.913 INFO:teuthology.orchestra.run.smithi145.stdout:2945 trunc from 0x9b22800 to 0x685bc00 2024-09-14T07:09:23.936 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:23.932+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:23.953 INFO:teuthology.orchestra.run.smithi145.stdout:2949 trunc from 0x685bc00 to 0x7348000 2024-09-14T07:09:23.987 INFO:teuthology.orchestra.run.smithi145.stdout:2951 trunc from 0x7348000 to 0x1c23800 2024-09-14T07:09:24.008 INFO:teuthology.orchestra.run.smithi145.stdout:2952 read 0x117ec00 thru 0x11823f5 (0x37f6 bytes) 2024-09-14T07:09:24.011 INFO:teuthology.orchestra.run.smithi145.stdout:2953 punch from 0x117c400 to 0x117fe00, (0x3a00 bytes) 2024-09-14T07:09:24.011 INFO:teuthology.orchestra.run.smithi145.stdout:2956 read 0x621000 thru 0x62df9b (0xcf9c bytes) 2024-09-14T07:09:24.011 INFO:teuthology.orchestra.run.smithi145.stdout:2961 clone 22 order 18 su 65536 sc 13 2024-09-14T07:09:24.024 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:24.024 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:24.037 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:24.032+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:24.038 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:24.138 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:24.132+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:24.167 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:24.167 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:24.183 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:24.239 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:24.236+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:24.320 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:09:24.339 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:24.336+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:24.358 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:24.358 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:24.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:24.373 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:24.425 INFO:teuthology.orchestra.run.smithi145.stdout:flattening image image_client.0-clone21 2024-09-14T07:09:24.442 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:24.436+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:24.515 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:24.515 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:24.533 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:24.533 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:24.543 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:24.540+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:24.550 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:24.553 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:24.553 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:24.567 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:24.569 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:24.588 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:24.589 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:24.606 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:24.606 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:24.667 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:24.664+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:24.691 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:24.691 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:24.691 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:24.691 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:24.705 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:24.755 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:24.752+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:24.835 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:24.835 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:24.836 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:24.836 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:24.836 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:24.849 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:24.844+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:24.850 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:24.902 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T07:09:24.950 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:24.944+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:24.971 INFO:teuthology.orchestra.run.smithi145.stdout:2964 punch from 0x130da00 to 0x1318000, (0xa600 bytes) 2024-09-14T07:09:25.013 INFO:teuthology.orchestra.run.smithi145.stdout:2965 write 0x2d49400 thru 0x2d565f2 (0xd1f3 bytes) 2024-09-14T07:09:25.015 INFO:teuthology.orchestra.run.smithi145.stdout:2966 read 0x732e00 thru 0x73f153 (0xc354 bytes) 2024-09-14T07:09:25.015 INFO:teuthology.orchestra.run.smithi145.stdout:2967 read 0x120400 thru 0x1207f4 (0x3f5 bytes) 2024-09-14T07:09:25.021 INFO:teuthology.orchestra.run.smithi145.stdout:2968 trunc from 0x2d56600 to 0x38ae400 2024-09-14T07:09:25.031 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:25.031 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:25.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:25.031 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:25.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.031 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.732152654000004, 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:25.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:25.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:25.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.037 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 54.872677066999998, 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:25.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 54.872574782000001, 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:25.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:25.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:25.047 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 54.173522941000002, 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:25.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:25.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688601749, 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:09:25.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688566528999999, 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:09:25.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:25.061 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688511380000001, 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:09:25.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.065 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688477311, 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.066 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:25.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688433965000002, 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.070 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:09:25.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688400441999999, 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:25.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:09:25.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688353192000001, 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.078 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:09:25.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.081 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688238943999998, 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:09:25.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:09:25.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:09:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688129248999999, 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.086 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:09:25.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.089 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.688063106000001, 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.090 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.093 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.687946602, 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:09:25.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.687880946, 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:09:25.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:09:25.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.687821626000002, 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.687680757999999, 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:09:25.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:09:25.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.687265651000001, 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:09:25.110 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:25.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.114 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.687197911999998, 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:25.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.118 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:25.120 INFO:teuthology.orchestra.run.smithi145.stdout:2969 write 0x4077c00 thru 0x4083e68 (0xc269 bytes) 2024-09-14T07:09:25.120 INFO:teuthology.orchestra.run.smithi145.stdout:2970 trunc from 0x4084000 to 0x360fe00 2024-09-14T07:09:25.120 INFO:teuthology.orchestra.run.smithi145.stdout:2971 write 0xfe0400 thru 0xfeadf9 (0xa9fa bytes) 2024-09-14T07:09:25.121 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:25.121 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:25.116+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:25.142 INFO:teuthology.orchestra.run.smithi145.stdout:2972 write 0x4391c00 thru 0x4391d3f (0x140 bytes) 2024-09-14T07:09:25.144 INFO:teuthology.orchestra.run.smithi145.stdout:2973 write 0x207c600 thru 0x207cdd2 (0x7d3 bytes) 2024-09-14T07:09:25.144 INFO:teuthology.orchestra.run.smithi145.stdout:2974 read 0x437d600 thru 0x438b96b (0xe36c bytes) 2024-09-14T07:09:25.210 INFO:teuthology.orchestra.run.smithi145.stdout:2975 write 0xa516400 thru 0xa52437e (0xdf7f bytes) 2024-09-14T07:09:25.211 INFO:teuthology.orchestra.run.smithi145.stdout:2977 punch from 0x65c5c00 to 0x65d3800, (0xdc00 bytes) 2024-09-14T07:09:25.212 INFO:teuthology.orchestra.run.smithi145.stdout:2978 read 0x68c600 thru 0x68d1fc (0xbfd bytes) 2024-09-14T07:09:25.212 INFO:teuthology.orchestra.run.smithi145.stdout:2979 write 0x4499000 thru 0x449af27 (0x1f28 bytes) 2024-09-14T07:09:25.212 INFO:teuthology.orchestra.run.smithi145.stdout:2980 read 0x7145a00 thru 0x7151671 (0xbc72 bytes) 2024-09-14T07:09:25.213 INFO:teuthology.orchestra.run.smithi145.stdout:2981 punch from 0x12cfe00 to 0x12d0c00, (0xe00 bytes) 2024-09-14T07:09:25.220 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:25.216+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:25.249 INFO:teuthology.orchestra.run.smithi145.stdout:2982 write 0xd7c0000 thru 0xd7ca7e2 (0xa7e3 bytes) 2024-09-14T07:09:25.251 INFO:teuthology.orchestra.run.smithi145.stdout:2984 read 0x3491600 thru 0x34939e5 (0x23e6 bytes) 2024-09-14T07:09:25.251 INFO:teuthology.orchestra.run.smithi145.stdout:2985 read 0x7612e00 thru 0x761d8f0 (0xaaf1 bytes) 2024-09-14T07:09:25.251 INFO:teuthology.orchestra.run.smithi145.stdout:2989 write 0x837d600 thru 0x838ce2b (0xf82c bytes) 2024-09-14T07:09:25.252 INFO:teuthology.orchestra.run.smithi145.stdout:2990 write 0x88aa800 thru 0x88afd57 (0x5558 bytes) 2024-09-14T07:09:25.252 INFO:teuthology.orchestra.run.smithi145.stdout:2992 read 0x9b26400 thru 0x9b30673 (0xa274 bytes) 2024-09-14T07:09:25.252 INFO:teuthology.orchestra.run.smithi145.stdout:2994 punch from 0x7410e00 to 0x7419c00, (0x8e00 bytes) 2024-09-14T07:09:25.252 INFO:teuthology.orchestra.run.smithi145.stdout:2996 punch from 0xe76800 to 0xe81400, (0xac00 bytes) 2024-09-14T07:09:25.252 INFO:teuthology.orchestra.run.smithi145.stdout:2998 write 0x60e9400 thru 0x60f36c2 (0xa2c3 bytes) 2024-09-14T07:09:25.253 INFO:teuthology.orchestra.run.smithi145.stdout:2999 write 0x6de2400 thru 0x6de9dcd (0x79ce bytes) 2024-09-14T07:09:25.253 INFO:teuthology.orchestra.run.smithi145.stdout:3001 read 0xd098c00 thru 0xd0a1706 (0x8b07 bytes) 2024-09-14T07:09:25.253 INFO:teuthology.orchestra.run.smithi145.stdout:3004 punch from 0x3d2e200 to 0x3d32400, (0x4200 bytes) 2024-09-14T07:09:25.253 INFO:teuthology.orchestra.run.smithi145.stdout:3006 punch from 0x5ad000 to 0x5b4200, (0x7200 bytes) 2024-09-14T07:09:25.255 INFO:teuthology.orchestra.run.smithi145.stdout:3009 trunc from 0xd7ca800 to 0xcd6e800 2024-09-14T07:09:25.293 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:09:25.314 INFO:teuthology.orchestra.run.smithi145.stdout:3010 write 0xdcb200 thru 0xdd9192 (0xdf93 bytes) 2024-09-14T07:09:25.316 INFO:teuthology.orchestra.run.smithi145.stdout:3011 read 0x441be00 thru 0x4428b11 (0xcd12 bytes) 2024-09-14T07:09:25.316 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:25.316 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:25.316 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:25.316 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:25.318 INFO:teuthology.orchestra.run.smithi145.stdout:3012 trunc from 0xcd6e800 to 0xc064400 2024-09-14T07:09:25.320 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:25.316+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:25.333 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:25.345 INFO:teuthology.orchestra.run.smithi145.stdout:3013 punch from 0x4d0d400 to 0x4d15000, (0x7c00 bytes) 2024-09-14T07:09:25.348 INFO:teuthology.orchestra.run.smithi145.stdout:3014 punch from 0x6769200 to 0x6771a00, (0x8800 bytes) 2024-09-14T07:09:25.348 INFO:teuthology.orchestra.run.smithi145.stdout:3015 write 0xae9cc00 thru 0xaeaadd1 (0xe1d2 bytes) 2024-09-14T07:09:25.349 INFO:teuthology.orchestra.run.smithi145.stdout:3018 read 0x1a5ae00 thru 0x1a5b49e (0x69f bytes) 2024-09-14T07:09:25.349 INFO:teuthology.orchestra.run.smithi145.stdout:3019 write 0x10a3200 thru 0x10b2115 (0xef16 bytes) 2024-09-14T07:09:25.350 INFO:teuthology.orchestra.run.smithi145.stdout:3020 write 0x5227c00 thru 0x5236dbb (0xf1bc bytes) 2024-09-14T07:09:25.351 INFO:teuthology.orchestra.run.smithi145.stdout:3021 read 0xa789600 thru 0xa78b5c0 (0x1fc1 bytes) 2024-09-14T07:09:25.351 INFO:teuthology.orchestra.run.smithi145.stdout:3023 read 0x3cefe00 thru 0x3cfa0ad (0xa2ae bytes) 2024-09-14T07:09:25.351 INFO:teuthology.orchestra.run.smithi145.stdout:3024 read 0x28e0600 thru 0x28e5204 (0x4c05 bytes) 2024-09-14T07:09:25.351 INFO:teuthology.orchestra.run.smithi145.stdout:3026 read 0x9f4ac00 thru 0x9f4e46e (0x386f bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3027 write 0xb051600 thru 0xb05d15e (0xbb5f bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3028 write 0x5dfb200 thru 0x5e038e0 (0x86e1 bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3034 read 0x2498a00 thru 0x249e217 (0x5818 bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3035 write 0x1777200 thru 0x177f54e (0x834f bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3037 write 0x1e40000 thru 0x1e43cca (0x3ccb bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3038 read 0x9b01200 thru 0x9b06021 (0x4e22 bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3039 read 0x2d77e00 thru 0x2d7c859 (0x4a5a bytes) 2024-09-14T07:09:25.352 INFO:teuthology.orchestra.run.smithi145.stdout:3040 punch from 0x4527a00 to 0x4528400, (0xa00 bytes) 2024-09-14T07:09:25.353 INFO:teuthology.orchestra.run.smithi145.stdout:3042 write 0xbb7f000 thru 0xbb7fced (0xcee bytes) 2024-09-14T07:09:25.353 INFO:teuthology.orchestra.run.smithi145.stdout:3044 read 0x831c200 thru 0x8320bda (0x49db bytes) 2024-09-14T07:09:25.353 INFO:teuthology.orchestra.run.smithi145.stdout:3045 read 0xa3f3e00 thru 0xa3fc08f (0x8290 bytes) 2024-09-14T07:09:25.359 INFO:teuthology.orchestra.run.smithi145.stdout:3046 trunc from 0xc064400 to 0x9676000 2024-09-14T07:09:25.401 INFO:teuthology.orchestra.run.smithi145.stdout:3049 trunc from 0x9676000 to 0x7e5d400 2024-09-14T07:09:25.421 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:25.416+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:25.422 INFO:teuthology.orchestra.run.smithi145.stdout:3050 read 0x7694200 thru 0x769fe9b (0xbc9c bytes) 2024-09-14T07:09:25.424 INFO:teuthology.orchestra.run.smithi145.stdout:3055 punch from 0x583e000 to 0x584bc00, (0xdc00 bytes) 2024-09-14T07:09:25.424 INFO:teuthology.orchestra.run.smithi145.stdout:3056 read 0x672e600 thru 0x673e2d9 (0xfcda bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3058 punch from 0x5cf4800 to 0x5cfd600, (0x8e00 bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3059 read 0x6c8800 thru 0x6cd154 (0x4955 bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3060 write 0x44f8200 thru 0x44fa3aa (0x21ab bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3062 write 0x7386400 thru 0x7393a44 (0xd645 bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3067 write 0x2951400 thru 0x295a7f3 (0x93f4 bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3068 read 0x5de0e00 thru 0x5de8354 (0x7555 bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3074 read 0x26ca400 thru 0x26cba91 (0x1692 bytes) 2024-09-14T07:09:25.425 INFO:teuthology.orchestra.run.smithi145.stdout:3076 write 0x350c00 thru 0x3587f2 (0x7bf3 bytes) 2024-09-14T07:09:25.431 INFO:teuthology.orchestra.run.smithi145.stdout:3077 trunc from 0x7e5d400 to 0x5a56400 2024-09-14T07:09:25.477 INFO:teuthology.orchestra.run.smithi145.stdout:3080 write 0x4a28c00 thru 0x4a2f3f4 (0x67f5 bytes) 2024-09-14T07:09:25.479 INFO:teuthology.orchestra.run.smithi145.stdout:3082 read 0x3e21800 thru 0x3e2a514 (0x8d15 bytes) 2024-09-14T07:09:25.504 INFO:teuthology.orchestra.run.smithi145.stdout:3083 write 0x6837600 thru 0x6846af7 (0xf4f8 bytes) 2024-09-14T07:09:25.504 INFO:teuthology.orchestra.run.smithi145.stdout:3086 flatten 2024-09-14T07:09:25.522 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:25.516+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:25.526 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:25.527 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:25.527 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:25.546 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:25.600 INFO:teuthology.orchestra.run.smithi145.stdout:3088 write 0x6996200 thru 0x699647f (0x280 bytes) 2024-09-14T07:09:25.601 INFO:teuthology.orchestra.run.smithi145.stdout:3093 write 0x5e0e00 thru 0x5e1ba2 (0xda3 bytes) 2024-09-14T07:09:25.607 INFO:teuthology.orchestra.run.smithi145.stdout:3095 trunc from 0x6996600 to 0x499d600 2024-09-14T07:09:25.622 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:25.616+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:25.639 INFO:teuthology.orchestra.run.smithi145.stdout:3096 read 0x3059600 thru 0x3066c99 (0xd69a bytes) 2024-09-14T07:09:25.641 INFO:teuthology.orchestra.run.smithi145.stdout:3097 read 0x22c9800 thru 0x22d7a1a (0xe21b bytes) 2024-09-14T07:09:25.667 INFO:teuthology.orchestra.run.smithi145.stdout:3099 write 0x906d000 thru 0x90743dc (0x73dd bytes) 2024-09-14T07:09:25.674 INFO:teuthology.orchestra.run.smithi145.stdout:3100 read 0x2fef000 thru 0x2ff7125 (0x8126 bytes) 2024-09-14T07:09:25.674 INFO:teuthology.orchestra.run.smithi145.stdout:3101 read 0x462c600 thru 0x4632067 (0x5a68 bytes) 2024-09-14T07:09:25.679 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:25.679 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 92.848354517000004, 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:25.680 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.684 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 92.847580168999997, 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:25.685 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:25.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:25.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:25.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.836397171000002, 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.539468964999998, 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.700 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.304565141000001, 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:25.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:09:25.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.304003349999999, 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:09:25.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:09:25.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.303508783000002, 2024-09-14T07:09:25.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:09:25.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.713 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.302891364000001, 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:09:25.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:09:25.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.302463105000001, 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.718 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.302067241, 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:09:25.722 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.726 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.301475565000001, 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:09:25.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:09:25.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.730 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.298146451000001, 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:25.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:09:25.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:09:25.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.298105400000001, 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.29805314, 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.739 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:09:25.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.742 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.298014253000002, 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:09:25.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:09:25.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.297954910000001, 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.297900821999999, 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.751 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.297861469000001, 2024-09-14T07:09:25.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:25.759 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.297823689000001, 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:25.763 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.297786985999998, 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:25.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:09:25.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:09:25.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:25.768 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:25.768 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:25.769 INFO:teuthology.orchestra.run.smithi145.stdout:3102 trunc from 0x9074400 to 0x29f4800 2024-09-14T07:09:25.769 INFO:teuthology.orchestra.run.smithi145.stdout:3104 read 0x1e02400 thru 0x1e11ecd (0xface bytes) 2024-09-14T07:09:25.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:25.770 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:25.764+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:25.792 INFO:teuthology.orchestra.run.smithi145.stdout:3106 write 0x9b56c00 thru 0x9b58ae3 (0x1ee4 bytes) 2024-09-14T07:09:25.793 INFO:teuthology.orchestra.run.smithi145.stdout:3107 write 0x10e5a00 thru 0x10ea31f (0x4920 bytes) 2024-09-14T07:09:25.794 INFO:teuthology.orchestra.run.smithi145.stdout:3108 punch from 0x36ba00 to 0x36e200, (0x2800 bytes) 2024-09-14T07:09:25.794 INFO:teuthology.orchestra.run.smithi145.stdout:3112 write 0x2746c00 thru 0x2753081 (0xc482 bytes) 2024-09-14T07:09:25.794 INFO:teuthology.orchestra.run.smithi145.stdout:3114 write 0x8853400 thru 0x88560c3 (0x2cc4 bytes) 2024-09-14T07:09:25.795 INFO:teuthology.orchestra.run.smithi145.stdout:3115 read 0x3c8e000 thru 0x3c91540 (0x3541 bytes) 2024-09-14T07:09:25.795 INFO:teuthology.orchestra.run.smithi145.stdout:3116 punch from 0x55f9e00 to 0x55fb200, (0x1400 bytes) 2024-09-14T07:09:25.796 INFO:teuthology.orchestra.run.smithi145.stdout:3117 read 0x4bd5600 thru 0x4bd8d45 (0x3746 bytes) 2024-09-14T07:09:25.796 INFO:teuthology.orchestra.run.smithi145.stdout:3120 write 0x7d78e00 thru 0x7d8447f (0xb680 bytes) 2024-09-14T07:09:25.796 INFO:teuthology.orchestra.run.smithi145.stdout:3121 read 0x44c8e00 thru 0x44d17d9 (0x89da bytes) 2024-09-14T07:09:25.796 INFO:teuthology.orchestra.run.smithi145.stdout:3122 read 0x3344000 thru 0x3349c3e (0x5c3f bytes) 2024-09-14T07:09:25.796 INFO:teuthology.orchestra.run.smithi145.stdout:3127 read 0x93b1000 thru 0x93bf99c (0xe99d bytes) 2024-09-14T07:09:25.858 INFO:teuthology.orchestra.run.smithi145.stdout:3130 write 0xd293e00 thru 0xd298413 (0x4614 bytes) 2024-09-14T07:09:25.861 INFO:teuthology.orchestra.run.smithi145.stdout:3131 read 0x1c4000 thru 0x1ca4dd (0x64de bytes) 2024-09-14T07:09:25.862 INFO:teuthology.orchestra.run.smithi145.stdout:3133 write 0xac25200 thru 0xac301a0 (0xafa1 bytes) 2024-09-14T07:09:25.862 INFO:teuthology.orchestra.run.smithi145.stdout:3135 write 0x22da400 thru 0x22e3c5c (0x985d bytes) 2024-09-14T07:09:25.863 INFO:teuthology.orchestra.run.smithi145.stdout:3136 read 0x115b600 thru 0x11619df (0x63e0 bytes) 2024-09-14T07:09:25.864 INFO:teuthology.orchestra.run.smithi145.stdout:3137 read 0x8a85400 thru 0x8a91996 (0xc597 bytes) 2024-09-14T07:09:25.870 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:25.864+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:25.875 INFO:teuthology.orchestra.run.smithi145.stdout:3138 trunc from 0xd298600 to 0x7d65600 2024-09-14T07:09:25.914 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:25.914 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:25.914 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:25.914 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:25.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:25.938 INFO:teuthology.orchestra.run.smithi145.stdout:3140 trunc from 0x7d65600 to 0x7bd3400 2024-09-14T07:09:25.971 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:25.968+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:25.990 INFO:teuthology.orchestra.run.smithi145.stdout:3141 trunc from 0x7bd3400 to 0xbc49000 2024-09-14T07:09:26.036 INFO:teuthology.orchestra.run.smithi145.stdout:3142 trunc from 0xbc49000 to 0x52e5000 2024-09-14T07:09:26.072 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:26.068+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:26.101 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:26.101 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:26.101 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:26.115 INFO:teuthology.orchestra.run.smithi145.stdout:3143 write 0x8f9d000 thru 0x8fabd6c (0xed6d bytes) 2024-09-14T07:09:26.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:26.119 INFO:teuthology.orchestra.run.smithi145.stdout:3144 read 0x2a91600 thru 0x2a9263c (0x103d bytes) 2024-09-14T07:09:26.119 INFO:teuthology.orchestra.run.smithi145.stdout:3146 punch from 0x2027200 to 0x202e400, (0x7200 bytes) 2024-09-14T07:09:26.120 INFO:teuthology.orchestra.run.smithi145.stdout:3147 write 0x437f400 thru 0x438ba96 (0xc697 bytes) 2024-09-14T07:09:26.120 INFO:teuthology.orchestra.run.smithi145.stdout:3149 read 0x439c000 thru 0x439f711 (0x3712 bytes) 2024-09-14T07:09:26.120 INFO:teuthology.orchestra.run.smithi145.stdout:3151 write 0x317ec00 thru 0x318c753 (0xdb54 bytes) 2024-09-14T07:09:26.120 INFO:teuthology.orchestra.run.smithi145.stdout:3154 punch from 0x120a800 to 0x1211a00, (0x7200 bytes) 2024-09-14T07:09:26.121 INFO:teuthology.orchestra.run.smithi145.stdout:3159 write 0x6fe2600 thru 0x6fed3f7 (0xadf8 bytes) 2024-09-14T07:09:26.134 INFO:teuthology.orchestra.run.smithi145.stdout:3160 trunc from 0x8fabe00 to 0x21d2200 2024-09-14T07:09:26.173 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:26.168+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:26.185 INFO:teuthology.orchestra.run.smithi145.stdout:3161 clone 23 order 20 su 262144 sc 9 2024-09-14T07:09:26.274 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:26.268+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.999940089000006, 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:26.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:26.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:26.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.970432458000005, 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:26.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:26.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:26.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:26.301 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.961785098999997, 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:26.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:26.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.295692759000005, 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:26.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.128963533000004, 2024-09-14T07:09:26.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:26.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:26.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.195962524999999, 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1413 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4405738039999996, 2024-09-14T07:09:26.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039824520000000002, 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1413 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-07 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848109+0000", 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:26.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848113+0000", 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6079999999999998e-06 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848116+0000", 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887648+0000", 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039532253000000003 2024-09-14T07:09:26.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887688+0000", 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9777000000000002e-05 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887932+0000", 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024362200000000001 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.323 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1414 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4405613850000005, 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040080681999999999, 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1414 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:26.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848121+0000", 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.122e-06 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848124+0000", 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.994e-06 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848126+0000", 2024-09-14T07:09:26.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5100000000000001e-06 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887949+0000", 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039822716000000001 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887988+0000", 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8862999999999997e-05 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888200+0000", 2024-09-14T07:09:26.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021226400000000001 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1415 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4405447739999993, 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040391931999999998, 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1415 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848137+0000", 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1e-07 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848139+0000", 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.052e-06 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848146+0000", 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3849999999999996e-06 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848150+0000", 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5420000000000001e-06 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888232+0000", 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040082568999999998 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888274+0000", 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1869000000000002e-05 2024-09-14T07:09:26.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888528+0000", 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025420500000000002 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4405193529999991, 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.336 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4405014309999995, 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:09:26.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4404886619999999, 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4404784859999999, 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:26.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:26.349 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4404682910000002, 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:26.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4404582710000007, 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.354 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:09:26.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:09:26.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1642 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 831488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3870474169999998, 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039801272999999998, 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1642 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901637+0000", 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4959999999999999e-06 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901643+0000", 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4710000000000004e-06 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901648+0000", 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941196+0000", 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039548176999999997 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941234+0000", 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7966999999999997e-05 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941435+0000", 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020081799999999999 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1643 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 854016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.362 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3870254360000001, 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040041757999999997, 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1643 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901658+0000", 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3190000000000002e-06 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901666+0000", 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3769999999999998e-06 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901671+0000", 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.006e-06 2024-09-14T07:09:26.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941457+0000", 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039786119000000002 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941496+0000", 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8742000000000003e-05 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941697+0000", 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201795 2024-09-14T07:09:26.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1644 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3870030789999994, 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040289206000000001, 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1644 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901681+0000", 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2170000000000001e-06 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901687+0000", 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8009999999999998e-06 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901693+0000", 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941720+0000", 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040027342 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941758+0000", 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8095e-05 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941967+0000", 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000209139 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1645 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3869807450000007, 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040527821999999998, 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1645 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901701+0000", 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901703+0000", 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.176e-06 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901711+0000", 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1519999999999992e-06 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901716+0000", 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0810000000000001e-06 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941990+0000", 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040274094000000003 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942027+0000", 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6726e-05 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942228+0000", 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020122599999999999 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3869571569999994, 2024-09-14T07:09:26.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:09:26.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:26.379 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:26.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:26.381 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:26.376+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:26.482 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:26.476+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:26.561 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:26.561 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:26.561 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:26.561 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:26.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:26.583 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:26.576+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:26.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:26.711 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:26.708+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:26.751 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:26.751 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:26.752 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:26.752 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:26.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:26.811 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:26.808+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:26.813 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:26.813 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:26.830 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:26.830 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:26.847 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:26.848 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:26.886 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:26.880+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:26.891 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:26.891 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:26.912 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:26.912 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:26.917 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T07:09:26.940 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:26.940 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:26.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:27.003 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:27.000+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:27.087 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:27.084+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:27.124 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:27.125 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:27.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:27.188 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:27.184+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:27.289 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:27.284+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:27.318 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:27.318 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:27.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:09:27.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:27.390 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:27.384+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:27.444 INFO:teuthology.orchestra.run.smithi145.stdout:3163 trunc from 0x21d2200 to 0x25f6200 2024-09-14T07:09:27.467 INFO:teuthology.orchestra.run.smithi145.stdout:3164 punch from 0x41fc00 to 0x427e00, (0x8200 bytes) 2024-09-14T07:09:27.469 INFO:teuthology.orchestra.run.smithi145.stdout:3165 read 0x814000 thru 0x8173ac (0x33ad bytes) 2024-09-14T07:09:27.491 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:27.484+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:27.496 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:27.497 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:27.511 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:27.591 INFO:teuthology.orchestra.run.smithi145.stdout:3166 write 0xdc91c00 thru 0xdc9b7e7 (0x9be8 bytes) 2024-09-14T07:09:27.592 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:27.588+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:27.593 INFO:teuthology.orchestra.run.smithi145.stdout:3171 punch from 0x1456000 to 0x1462400, (0xc400 bytes) 2024-09-14T07:09:27.594 INFO:teuthology.orchestra.run.smithi145.stdout:3174 write 0x5ad6800 thru 0x5adec43 (0x8444 bytes) 2024-09-14T07:09:27.594 INFO:teuthology.orchestra.run.smithi145.stdout:3176 clone 24 order 18 su 131072 sc 5 2024-09-14T07:09:27.649 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:27.649 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:27.662 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:27.711 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:27.708+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:27.780 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:27.781 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:27.793 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:27.788+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:27.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:27.894 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:27.888+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:27.959 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:27.959 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:27.974 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:28.019 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:28.016+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:28.096 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:28.092+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:28.140 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:28.140 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:28.154 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:28.200 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:28.196+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:28.276 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:28.276 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:28.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:28.294 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:09:28.294 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:28.295 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T07:09:28.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T07:09:28.313 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:09:28.331 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:28.328+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:28.399 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:28.396+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:28.438 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone23 from 0xdc9b800 (overlap 0x21d2200) to 0x1e70400 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-09-14T07:09:28.467 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-09-14T07:09:28.468 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-09-14T07:09:28.468 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:28.468 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:28.470 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:09:28.470 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:09:28.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T07:09:28.483 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:28.527 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:28.524+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:28.601 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:28.596+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:28.609 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:28.609 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:28.625 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:09:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:09:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:09:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:09:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:09:28.679 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:09:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-09-14T07:09:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:09:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-09-14T07:09:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-09-14T07:09:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-09-14T07:09:28.680 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-09-14T07:09:28.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:09:28.683 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:09:28.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T07:09:28.727 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:28.724+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:28.788 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:28.788 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:28.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:28.803 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:28.796+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:09:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-09-14T07:09:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-09-14T07:09:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-09-14T07:09:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-09-14T07:09:28.849 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:09:28.849 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:09:28.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T07:09:28.915 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T07:09:28.935 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:28.932+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:28.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:28.982 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:28.982 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:28.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:29.047 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:29.044+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-09-14T07:09:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-09-14T07:09:29.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:09:29.082 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:09:29.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T07:09:29.090 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:29.107 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:29.107 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:29.108 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:29.104+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:29.125 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:29.126 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:29.138 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:29.139 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:29.139 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:29.139 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:29.147 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:29.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:29.168 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:29.168 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:29.189 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:29.189 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:29.205 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:29.200+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:29.207 INFO:teuthology.orchestra.run.smithi145.stdout:3177 trunc from 0xdc9b800 to 0xd082c00 2024-09-14T07:09:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:09:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:09:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:09:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:09:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:09:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:09:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-09-14T07:09:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:09:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-09-14T07:09:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-09-14T07:09:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-09-14T07:09:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-09-14T07:09:29.243 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:09:29.243 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:09:29.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T07:09:29.249 INFO:teuthology.orchestra.run.smithi145.stdout:3180 write 0x5c6bc00 thru 0x5c6cba7 (0xfa8 bytes) 2024-09-14T07:09:29.265 INFO:teuthology.orchestra.run.smithi145.stdout:3183 trunc from 0xd082c00 to 0xc179000 2024-09-14T07:09:29.285 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:29.285 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:29.286 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:29.286 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:29.301 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:29.323 INFO:teuthology.orchestra.run.smithi145.stdout:3187 read 0xa07e000 thru 0xa082a44 (0x4a45 bytes) 2024-09-14T07:09:29.347 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:29.344+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:29.371 INFO:teuthology.orchestra.run.smithi145.stdout:3190 write 0xec4b400 thru 0xec5682c (0xb42d bytes) 2024-09-14T07:09:29.372 INFO:teuthology.orchestra.run.smithi145.stdout:3192 punch from 0xd5ca00 to 0xd5e000, (0x1600 bytes) 2024-09-14T07:09:29.373 INFO:teuthology.orchestra.run.smithi145.stdout:3193 read 0x523d800 thru 0x5245660 (0x7e61 bytes) 2024-09-14T07:09:29.408 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:29.404+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-09-14T07:09:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-09-14T07:09:29.431 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-09-14T07:09:29.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:09:29.434 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:09:29.439 INFO:teuthology.orchestra.run.smithi145.stdout:3194 trunc from 0xec56a00 to 0x1b90e00 2024-09-14T07:09:29.464 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:29.465 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:29.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:29.465 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:29.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.465 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:29.465 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:29.465 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 72.175229799999997, 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:29.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.471 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 72.145722168999995, 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 72.137074810000001, 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:29.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 67.470982469999996, 2024-09-14T07:09:29.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:29.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:29.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 67.304253243999995, 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:29.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:29.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.371252235999997, 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:29.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:29.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1413 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.494 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615863515000001, 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039824520000000002, 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1413 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-07 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848109+0000", 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848113+0000", 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6079999999999998e-06 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848116+0000", 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:29.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887648+0000", 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039532253000000003 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887688+0000", 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9777000000000002e-05 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887932+0000", 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024362200000000001 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.498 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1414 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615851096, 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040080681999999999, 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1414 2024-09-14T07:09:29.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848121+0000", 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.122e-06 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848124+0000", 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.994e-06 2024-09-14T07:09:29.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848126+0000", 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5100000000000001e-06 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887949+0000", 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039822716000000001 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887988+0000", 2024-09-14T07:09:29.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8862999999999997e-05 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888200+0000", 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021226400000000001 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1415 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615834485000001, 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040391931999999998, 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1415 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848137+0000", 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1e-07 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848139+0000", 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.052e-06 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848146+0000", 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3849999999999996e-06 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848150+0000", 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5420000000000001e-06 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888232+0000", 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040082568999999998 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888274+0000", 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1869000000000002e-05 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888528+0000", 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025420500000000002 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615809064, 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:09:29.507 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615791141999999, 2024-09-14T07:09:29.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:09:29.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:09:29.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.515 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615778372999999, 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:09:29.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:09:29.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615768197, 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:09:29.520 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:29.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615758002, 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:29.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:09:29.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:09:29.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.532 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.532 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.532 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:29.532 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.615747982, 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:09:29.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.534 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:29.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:09:29.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:09:29.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:09:29.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:09:29.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.539 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1642 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 831488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.562337127999999, 2024-09-14T07:09:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039801272999999998, 2024-09-14T07:09:29.540 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1642 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.541 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:29.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901637+0000", 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4959999999999999e-06 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901643+0000", 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4710000000000004e-06 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901648+0000", 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941196+0000", 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039548176999999997 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941234+0000", 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7966999999999997e-05 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941435+0000", 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020081799999999999 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1643 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 854016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.562315147, 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040041757999999997, 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1643 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901658+0000", 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3190000000000002e-06 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901666+0000", 2024-09-14T07:09:29.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3769999999999998e-06 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901671+0000", 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.006e-06 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941457+0000", 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039786119000000002 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941496+0000", 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8742000000000003e-05 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941697+0000", 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201795 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1644 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.562292790000001, 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040289206000000001, 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1644 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901681+0000", 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2170000000000001e-06 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901687+0000", 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8009999999999998e-06 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901693+0000", 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941720+0000", 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040027342 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941758+0000", 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8095e-05 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941967+0000", 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000209139 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.554 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1645 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.562270456, 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040527821999999998, 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1645 2024-09-14T07:09:29.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901701+0000", 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901703+0000", 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.176e-06 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901711+0000", 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1519999999999992e-06 2024-09-14T07:09:29.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901716+0000", 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0810000000000001e-06 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941990+0000", 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040274094000000003 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942027+0000", 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6726e-05 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942228+0000", 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020122599999999999 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.562246868000001, 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:29.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:29.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:29.564 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:3195 read 0x1ab0400 thru 0x1ab869f (0x82a0 bytes) 2024-09-14T07:09:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:3196 trunc from 0x1b90e00 to 0x1a3c600 2024-09-14T07:09:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:3198 trunc from 0x1a3c600 to 0x2690600 2024-09-14T07:09:29.566 INFO:teuthology.orchestra.run.smithi145.stdout:3201 write 0x46600 thru 0x48911 (0x2312 bytes) 2024-09-14T07:09:29.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:29.567 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:29.560+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:29.618 INFO:teuthology.orchestra.run.smithi145.stdout:3203 write 0xce49a00 thru 0xce53aff (0xa100 bytes) 2024-09-14T07:09:29.619 INFO:teuthology.orchestra.run.smithi145.stdout:3204 write 0x17d5800 thru 0x17daa26 (0x5227 bytes) 2024-09-14T07:09:29.619 INFO:teuthology.orchestra.run.smithi145.stdout:3205 punch from 0x664fc00 to 0x6653400, (0x3800 bytes) 2024-09-14T07:09:29.620 INFO:teuthology.orchestra.run.smithi145.stdout:3207 write 0x4084a00 thru 0x4084e96 (0x497 bytes) 2024-09-14T07:09:29.620 INFO:teuthology.orchestra.run.smithi145.stdout:3208 punch from 0x676ea00 to 0x6771a00, (0x3000 bytes) 2024-09-14T07:09:29.620 INFO:teuthology.orchestra.run.smithi145.stdout:3210 write 0x521ce00 thru 0x5226eee (0xa0ef bytes) 2024-09-14T07:09:29.620 INFO:teuthology.orchestra.run.smithi145.stdout:3211 write 0x6b9e00 thru 0x6c6211 (0xc412 bytes) 2024-09-14T07:09:29.625 INFO:teuthology.orchestra.run.smithi145.stdout:3212 trunc from 0xce53c00 to 0xab6aa00 2024-09-14T07:09:29.666 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:29.660+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:29.682 INFO:teuthology.orchestra.run.smithi145.stdout:3213 write 0x2bc9a00 thru 0x2bd923b (0xf83c bytes) 2024-09-14T07:09:29.700 INFO:teuthology.orchestra.run.smithi145.stdout:3214 trunc from 0xab6aa00 to 0x4e40a00 2024-09-14T07:09:29.743 INFO:teuthology.orchestra.run.smithi145.stdout:3215 punch from 0x136e400 to 0x136fe00, (0x1a00 bytes) 2024-09-14T07:09:29.744 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:29.745 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:29.745 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:29.749 INFO:teuthology.orchestra.run.smithi145.stdout:3216 trunc from 0x4e40a00 to 0x65d0c00 2024-09-14T07:09:29.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:29.773 INFO:teuthology.orchestra.run.smithi145.stdout:3217 punch from 0x4d6800 to 0x4d7400, (0xc00 bytes) 2024-09-14T07:09:29.775 INFO:teuthology.orchestra.run.smithi145.stdout:3218 read 0x51f3800 thru 0x5201575 (0xdd76 bytes) 2024-09-14T07:09:29.803 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:29.800+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:29.815 INFO:teuthology.orchestra.run.smithi145.stdout:3220 write 0xde37a00 thru 0xde441a8 (0xc7a9 bytes) 2024-09-14T07:09:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:3222 write 0x6c30a00 thru 0x6c36d00 (0x6301 bytes) 2024-09-14T07:09:29.818 INFO:teuthology.orchestra.run.smithi145.stdout:3225 read 0xb96c000 thru 0xb97a8f5 (0xe8f6 bytes) 2024-09-14T07:09:29.819 INFO:teuthology.orchestra.run.smithi145.stdout:3226 read 0x2e22000 thru 0x2e314a2 (0xf4a3 bytes) 2024-09-14T07:09:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:3228 write 0x6e0400 thru 0x6e50f3 (0x4cf4 bytes) 2024-09-14T07:09:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:3230 read 0x8c10c00 thru 0x8c1c2d2 (0xb6d3 bytes) 2024-09-14T07:09:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:3231 read 0xd731400 thru 0xd7386b6 (0x72b7 bytes) 2024-09-14T07:09:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:3238 read 0x92f8e00 thru 0x9305fe6 (0xd1e7 bytes) 2024-09-14T07:09:29.820 INFO:teuthology.orchestra.run.smithi145.stdout:3239 punch from 0xcb23000 to 0xcb2ce00, (0x9e00 bytes) 2024-09-14T07:09:29.821 INFO:teuthology.orchestra.run.smithi145.stdout:3240 trunc from 0xde44200 to 0xdaffa00 2024-09-14T07:09:29.867 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:29.864+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:29.872 INFO:teuthology.orchestra.run.smithi145.stdout:3245 read 0x5a3f000 thru 0x5a4ec8f (0xfc90 bytes) 2024-09-14T07:09:29.873 INFO:teuthology.orchestra.run.smithi145.stdout:3248 write 0xa2d3400 thru 0xa2dbe93 (0x8a94 bytes) 2024-09-14T07:09:29.874 INFO:teuthology.orchestra.run.smithi145.stdout:3250 write 0x27c2c00 thru 0x27c61fc (0x35fd bytes) 2024-09-14T07:09:29.874 INFO:teuthology.orchestra.run.smithi145.stdout:3253 read 0x7e5c00 thru 0x7ed885 (0x7c86 bytes) 2024-09-14T07:09:29.891 INFO:teuthology.orchestra.run.smithi145.stdout:3255 trunc from 0xdaffa00 to 0x3f22800 2024-09-14T07:09:29.936 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:29.936 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:29.936 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:29.951 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:29.962 INFO:teuthology.orchestra.run.smithi145.stdout:3256 trunc from 0x3f22800 to 0xabbc00 2024-09-14T07:09:29.969 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:29.964+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:30.015 INFO:teuthology.orchestra.run.smithi145.stdout:3259 trunc from 0xabbc00 to 0x8edc00 2024-09-14T07:09:30.062 INFO:teuthology.orchestra.run.smithi145.stdout:3261 read 0x10c200 thru 0x11970e (0xd50f bytes) 2024-09-14T07:09:30.064 INFO:teuthology.orchestra.run.smithi145.stdout:3262 punch from 0x1ed400 to 0x1fbc00, (0xe800 bytes) 2024-09-14T07:09:30.068 INFO:teuthology.orchestra.run.smithi145.stdout:3263 trunc from 0x8edc00 to 0x519600 2024-09-14T07:09:30.069 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:30.064+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:30.114 INFO:teuthology.orchestra.run.smithi145.stdout:3264 read 0x403e00 thru 0x40f40c (0xb60d bytes) 2024-09-14T07:09:30.117 INFO:teuthology.orchestra.run.smithi145.stdout:3265 read 0x345c00 thru 0x352af0 (0xcef1 bytes) 2024-09-14T07:09:30.119 INFO:teuthology.orchestra.run.smithi145.stdout:3266 read 0x282200 thru 0x2907a1 (0xe5a2 bytes) 2024-09-14T07:09:30.131 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:30.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:30.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.448916768999993, 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057940932000001, 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:09:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057911134000001, 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:09:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057878858999999, 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:09:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057820756000002, 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:09:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:09:30.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:09:30.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057779854, 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.159 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.163 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057718046000002, 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:09:30.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:09:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:09:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:09:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:09:30.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:09:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:09:30.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057683730000001, 2024-09-14T07:09:30.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.174 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:09:30.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:09:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:09:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:30.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057665059000001, 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:09:30.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057641022999999, 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:09:30.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:09:30.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057620989, 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:09:30.188 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:09:30.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057606153999998, 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057591528, 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:09:30.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:09:30.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057577027000001, 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:30.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:09:30.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057559271999999, 2024-09-14T07:09:30.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:30.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:09:30.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.208 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057544377999999, 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:09:30.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:09:30.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057530267000001, 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:09:30.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057508338000002, 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.220 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.057487027000001, 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:09:30.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.770136126000001, 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.225 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:09:30.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:09:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.230 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.230 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:30.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:30.233 INFO:teuthology.orchestra.run.smithi145.stdout:3268 write 0x10e8000 thru 0x10f3573 (0xb574 bytes) 2024-09-14T07:09:30.233 INFO:teuthology.orchestra.run.smithi145.stdout:3269 trunc from 0x10f3600 to 0xe14e800 2024-09-14T07:09:30.233 INFO:teuthology.orchestra.run.smithi145.stdout:3270 read 0x695c200 thru 0x696475a (0x855b bytes) 2024-09-14T07:09:30.233 INFO:teuthology.orchestra.run.smithi145.stdout:3272 read 0xb671800 thru 0xb680fe3 (0xf7e4 bytes) 2024-09-14T07:09:30.233 INFO:teuthology.orchestra.run.smithi145.stdout:3273 write 0xbb70400 thru 0xbb7b8c0 (0xb4c1 bytes) 2024-09-14T07:09:30.233 INFO:teuthology.orchestra.run.smithi145.stdout:3275 read 0x91f8600 thru 0x92002dd (0x7cde bytes) 2024-09-14T07:09:30.233 INFO:teuthology.orchestra.run.smithi145.stdout:3276 read 0x6d11e00 thru 0x6d1fafa (0xdcfb bytes) 2024-09-14T07:09:30.234 INFO:teuthology.orchestra.run.smithi145.stdout:3278 write 0x4c94200 thru 0x4c9c2c2 (0x80c3 bytes) 2024-09-14T07:09:30.234 INFO:teuthology.orchestra.run.smithi145.stdout:3279 punch from 0x623d000 to 0x623da00, (0xa00 bytes) 2024-09-14T07:09:30.234 INFO:teuthology.orchestra.run.smithi145.stdout:3280 write 0xa924a00 thru 0xa929bcb (0x51cc bytes) 2024-09-14T07:09:30.234 INFO:teuthology.orchestra.run.smithi145.stdout:3283 punch from 0x4857000 to 0x485d600, (0x6600 bytes) 2024-09-14T07:09:30.234 INFO:teuthology.orchestra.run.smithi145.stdout:3284 trunc from 0xe14e800 to 0x9573000 2024-09-14T07:09:30.234 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:30.228+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:30.287 INFO:teuthology.orchestra.run.smithi145.stdout:3285 trunc from 0x9573000 to 0xdce6800 2024-09-14T07:09:30.332 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:30.328+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:30.336 INFO:teuthology.orchestra.run.smithi145.stdout:3286 trunc from 0xdce6800 to 0x2445a00 2024-09-14T07:09:30.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:09:30.384 INFO:teuthology.orchestra.run.smithi145.stdout:3287 read 0x82400 thru 0x91e5f (0xfa60 bytes) 2024-09-14T07:09:30.393 INFO:teuthology.orchestra.run.smithi145.stdout:3289 trunc from 0x2445a00 to 0x49dbe00 2024-09-14T07:09:30.410 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:30.411 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:30.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:30.433 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:30.428+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:30.444 INFO:teuthology.orchestra.run.smithi145.stdout:3290 write 0x60cca00 thru 0x60d85e9 (0xbbea bytes) 2024-09-14T07:09:30.445 INFO:teuthology.orchestra.run.smithi145.stdout:3293 punch from 0x1743000 to 0x1752e00, (0xfe00 bytes) 2024-09-14T07:09:30.453 INFO:teuthology.orchestra.run.smithi145.stdout:3296 trunc from 0x60d8600 to 0x9876800 2024-09-14T07:09:30.479 INFO:teuthology.orchestra.run.smithi145.stdout:3297 write 0x191f400 thru 0x192484b (0x544c bytes) 2024-09-14T07:09:30.482 INFO:teuthology.orchestra.run.smithi145.stdout:3299 write 0x16cf200 thru 0x16d5619 (0x641a bytes) 2024-09-14T07:09:30.482 INFO:teuthology.orchestra.run.smithi145.stdout:3300 read 0x9657a00 thru 0x965e3a8 (0x69a9 bytes) 2024-09-14T07:09:30.493 INFO:teuthology.orchestra.run.smithi145.stdout:3301 trunc from 0x9876800 to 0x3b8ca00 2024-09-14T07:09:30.534 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:30.528+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:30.549 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:30.549 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:30.549 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:30.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:30.573 INFO:teuthology.orchestra.run.smithi145.stdout:3303 write 0x3fcba00 thru 0x3fd38b1 (0x7eb2 bytes) 2024-09-14T07:09:30.575 INFO:teuthology.orchestra.run.smithi145.stdout:3304 read 0xb39400 thru 0xb3d611 (0x4212 bytes) 2024-09-14T07:09:30.632 INFO:teuthology.orchestra.run.smithi145.stdout:3305 write 0xe34b600 thru 0xe358b05 (0xd506 bytes) 2024-09-14T07:09:30.635 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:30.628+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:30.635 INFO:teuthology.orchestra.run.smithi145.stdout:3306 read 0x2974600 thru 0x297ecd2 (0xa6d3 bytes) 2024-09-14T07:09:30.635 INFO:teuthology.orchestra.run.smithi145.stdout:3310 write 0x4a62000 thru 0x4a64c98 (0x2c99 bytes) 2024-09-14T07:09:30.644 INFO:teuthology.orchestra.run.smithi145.stdout:3311 trunc from 0xe358c00 to 0xaf05200 2024-09-14T07:09:30.688 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.386078964000006, 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:30.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:09:30.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:09:30.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.341088658000004, 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:30.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:30.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:09:30.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:30.701 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.039156202000001, 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:09:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:09:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:09:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:09:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:09:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:09:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:09:30.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:09:30.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.717828467999993, 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:09:30.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.715 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.280700069000002, 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:30.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.720 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.280689411000001, 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:09:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:09:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.725 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.280674396999999, 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:30.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:30.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.730 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.280663482000001, 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.280651972000001, 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.280639429000001, 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:09:30.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:30.743 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273803427000001, 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.744 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:09:30.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273792833000002, 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:09:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:30.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:09:30.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:09:30.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273745051999999, 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:30.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:09:30.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273731585, 2024-09-14T07:09:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:09:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:09:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273720612999998, 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:09:30.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:09:30.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:30.765 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273709851, 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:09:30.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:09:30.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273693797, 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:09:30.771 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:09:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273677226, 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:09:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273665449999999, 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.273654773000001, 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:30.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:09:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:09:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:09:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:09:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:30.790 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:30.792 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:30.793 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:30.788+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3313 trunc from 0xaf05200 to 0xe484400 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3315 read 0x4859a00 thru 0x486436a (0xa96b bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3316 read 0x6884600 thru 0x688b584 (0x6f85 bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3319 punch from 0x5020a00 to 0x5027800, (0x6e00 bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3320 write 0xa9e7000 thru 0xa9f1b7c (0xab7d bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3321 punch from 0x9c8c200 to 0x9c99a00, (0xd800 bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3322 read 0x102da00 thru 0x10350ae (0x76af bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3323 write 0x918f400 thru 0x919b547 (0xc148 bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3327 read 0x85a6e00 thru 0x85b0daa (0x9fab bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3328 write 0xe3e7400 thru 0xe3f566b (0xe26c bytes) 2024-09-14T07:09:30.795 INFO:teuthology.orchestra.run.smithi145.stdout:3330 write 0x6edd600 thru 0x6ee02b1 (0x2cb2 bytes) 2024-09-14T07:09:30.796 INFO:teuthology.orchestra.run.smithi145.stdout:3332 trunc from 0xe484400 to 0xc6ac800 2024-09-14T07:09:30.796 INFO:teuthology.orchestra.run.smithi145.stdout:3333 punch from 0x3fe7000 to 0x3fec800, (0x5800 bytes) 2024-09-14T07:09:30.796 INFO:teuthology.orchestra.run.smithi145.stdout:3337 read 0x4205a00 thru 0x420be35 (0x6436 bytes) 2024-09-14T07:09:30.796 INFO:teuthology.orchestra.run.smithi145.stdout:3339 write 0x3ad3c00 thru 0x3ad4aa5 (0xea6 bytes) 2024-09-14T07:09:30.796 INFO:teuthology.orchestra.run.smithi145.stdout:3340 write 0x7c08a00 thru 0x7c11e0d (0x940e bytes) 2024-09-14T07:09:30.797 INFO:teuthology.orchestra.run.smithi145.stdout:3343 read 0x5074200 thru 0x50797dd (0x55de bytes) 2024-09-14T07:09:30.797 INFO:teuthology.orchestra.run.smithi145.stdout:3344 write 0xab7a00 thru 0xaba438 (0x2a39 bytes) 2024-09-14T07:09:30.797 INFO:teuthology.orchestra.run.smithi145.stdout:3346 write 0x62d3c00 thru 0x62e396b (0xfd6c bytes) 2024-09-14T07:09:30.797 INFO:teuthology.orchestra.run.smithi145.stdout:3347 trunc from 0xc6ac800 to 0xeb3ac00 2024-09-14T07:09:30.825 INFO:teuthology.orchestra.run.smithi145.stdout:3352 read 0x1cbc200 thru 0x1cc5dd8 (0x9bd9 bytes) 2024-09-14T07:09:30.827 INFO:teuthology.orchestra.run.smithi145.stdout:3353 write 0x8cb0400 thru 0x8cb16e6 (0x12e7 bytes) 2024-09-14T07:09:30.828 INFO:teuthology.orchestra.run.smithi145.stdout:3354 read 0x6428c00 thru 0x642d265 (0x4666 bytes) 2024-09-14T07:09:30.828 INFO:teuthology.orchestra.run.smithi145.stdout:3355 read 0x6c86600 thru 0x6c94fcd (0xe9ce bytes) 2024-09-14T07:09:30.830 INFO:teuthology.orchestra.run.smithi145.stdout:3358 read 0x5581200 thru 0x5584499 (0x329a bytes) 2024-09-14T07:09:30.830 INFO:teuthology.orchestra.run.smithi145.stdout:3359 punch from 0x83d1a00 to 0x83d8a00, (0x7000 bytes) 2024-09-14T07:09:30.830 INFO:teuthology.orchestra.run.smithi145.stdout:3360 read 0xce8a00 thru 0xcebbf4 (0x31f5 bytes) 2024-09-14T07:09:30.830 INFO:teuthology.orchestra.run.smithi145.stdout:3362 read 0xd3dd600 thru 0xd3de518 (0xf19 bytes) 2024-09-14T07:09:30.830 INFO:teuthology.orchestra.run.smithi145.stdout:3365 write 0x620f800 thru 0x6212081 (0x2882 bytes) 2024-09-14T07:09:30.831 INFO:teuthology.orchestra.run.smithi145.stdout:3366 write 0x7edf200 thru 0x7ee220e (0x300f bytes) 2024-09-14T07:09:30.831 INFO:teuthology.orchestra.run.smithi145.stdout:3371 write 0x8d62800 thru 0x8d70d54 (0xe555 bytes) 2024-09-14T07:09:30.831 INFO:teuthology.orchestra.run.smithi145.stdout:3372 write 0xeb14800 thru 0xeb18b5b (0x435c bytes) 2024-09-14T07:09:30.854 INFO:teuthology.orchestra.run.smithi145.stdout:3373 trunc from 0xeb3ac00 to 0x2072600 2024-09-14T07:09:30.892 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:30.888+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:30.904 INFO:teuthology.orchestra.run.smithi145.stdout:3374 punch from 0x1ee2600 to 0x1ee8800, (0x6200 bytes) 2024-09-14T07:09:30.927 INFO:teuthology.orchestra.run.smithi145.stdout:3375 write 0x54c5e00 thru 0x54c9b7f (0x3d80 bytes) 2024-09-14T07:09:30.928 INFO:teuthology.orchestra.run.smithi145.stdout:3377 punch from 0x52600 to 0x54200, (0x1c00 bytes) 2024-09-14T07:09:30.929 INFO:teuthology.orchestra.run.smithi145.stdout:3380 write 0x1126400 thru 0x112da60 (0x7661 bytes) 2024-09-14T07:09:30.929 INFO:teuthology.orchestra.run.smithi145.stdout:3381 read 0x544a00 thru 0x54a7ad (0x5dae bytes) 2024-09-14T07:09:30.929 INFO:teuthology.orchestra.run.smithi145.stdout:3382 read 0x38d8400 thru 0x38e0494 (0x8095 bytes) 2024-09-14T07:09:30.940 INFO:teuthology.orchestra.run.smithi145.stdout:3384 trunc from 0x54c9c00 to 0x9ad5200 2024-09-14T07:09:30.952 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:30.952 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:30.952 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:30.952 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:30.956 INFO:teuthology.orchestra.run.smithi145.stdout:3385 read 0x10c2200 thru 0x10c9ff2 (0x7df3 bytes) 2024-09-14T07:09:30.959 INFO:teuthology.orchestra.run.smithi145.stdout:3386 trunc from 0x9ad5200 to 0xa96e400 2024-09-14T07:09:30.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:30.970 INFO:teuthology.orchestra.run.smithi145.stdout:3389 write 0xe73a00 thru 0xe7b7cd (0x7dce bytes) 2024-09-14T07:09:30.971 INFO:teuthology.orchestra.run.smithi145.stdout:3390 write 0x3e1e200 thru 0x3e2b325 (0xd126 bytes) 2024-09-14T07:09:30.971 INFO:teuthology.orchestra.run.smithi145.stdout:3392 punch from 0x5eea200 to 0x5ef9e00, (0xfc00 bytes) 2024-09-14T07:09:30.972 INFO:teuthology.orchestra.run.smithi145.stdout:3393 punch from 0x6000600 to 0x6005600, (0x5000 bytes) 2024-09-14T07:09:30.993 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:30.988+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:31.000 INFO:teuthology.orchestra.run.smithi145.stdout:3394 write 0xe097a00 thru 0xe09d1a8 (0x57a9 bytes) 2024-09-14T07:09:31.023 INFO:teuthology.orchestra.run.smithi145.stdout:3395 trunc from 0xe09d200 to 0x149c600 2024-09-14T07:09:31.060 INFO:teuthology.orchestra.run.smithi145.stdout:3399 punch from 0x8cf800 to 0x8d0400, (0xc00 bytes) 2024-09-14T07:09:31.060 INFO:teuthology.orchestra.run.smithi145.stdout:3401 read 0xa8ea00 thru 0xa9b22f (0xc830 bytes) 2024-09-14T07:09:31.060 INFO:teuthology.orchestra.run.smithi145.stdout:3406 read 0x4d9400 thru 0x4e2caa (0x98ab bytes) 2024-09-14T07:09:31.080 INFO:teuthology.orchestra.run.smithi145.stdout:3407 write 0x3456600 thru 0x345793a (0x133b bytes) 2024-09-14T07:09:31.081 INFO:teuthology.orchestra.run.smithi145.stdout:3410 read 0x1f3a00 thru 0x1f4fd5 (0x15d6 bytes) 2024-09-14T07:09:31.083 INFO:teuthology.orchestra.run.smithi145.stdout:3411 punch from 0x14b1c00 to 0x14bb600, (0x9a00 bytes) 2024-09-14T07:09:31.083 INFO:teuthology.orchestra.run.smithi145.stdout:3412 read 0x57000 thru 0x63ec4 (0xcec5 bytes) 2024-09-14T07:09:31.094 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:31.088+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:31.107 INFO:teuthology.orchestra.run.smithi145.stdout:3413 write 0x822a200 thru 0x82398c1 (0xf6c2 bytes) 2024-09-14T07:09:31.110 INFO:teuthology.orchestra.run.smithi145.stdout:3414 read 0x74b1400 thru 0x74b7770 (0x6371 bytes) 2024-09-14T07:09:31.110 INFO:teuthology.orchestra.run.smithi145.stdout:3415 write 0x79ea00 thru 0x7ac570 (0xdb71 bytes) 2024-09-14T07:09:31.110 INFO:teuthology.orchestra.run.smithi145.stdout:3420 punch from 0x2f9ea00 to 0x2fa2200, (0x3800 bytes) 2024-09-14T07:09:31.110 INFO:teuthology.orchestra.run.smithi145.stdout:3421 read 0x3674800 thru 0x3679214 (0x4a15 bytes) 2024-09-14T07:09:31.138 INFO:teuthology.orchestra.run.smithi145.stdout:3422 write 0xe0d0000 thru 0xe0d72d8 (0x72d9 bytes) 2024-09-14T07:09:31.139 INFO:teuthology.orchestra.run.smithi145.stdout:3423 read 0x8d03000 thru 0x8d096ec (0x66ed bytes) 2024-09-14T07:09:31.149 INFO:teuthology.orchestra.run.smithi145.stdout:3424 trunc from 0xe0d7400 to 0x9e91200 2024-09-14T07:09:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:31.153 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:31.153 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:31.154 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:31.154 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:31.169 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:31.172 INFO:teuthology.orchestra.run.smithi145.stdout:3429 read 0x2f81200 thru 0x2f83767 (0x2568 bytes) 2024-09-14T07:09:31.183 INFO:teuthology.orchestra.run.smithi145.stdout:3431 trunc from 0x9e91200 to 0x4b05200 2024-09-14T07:09:31.200 INFO:teuthology.orchestra.run.smithi145.stdout:3432 read 0x37c4200 thru 0x37d3186 (0xef87 bytes) 2024-09-14T07:09:31.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:31.215 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:31.212+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:31.239 INFO:teuthology.orchestra.run.smithi145.stdout:3435 write 0xed74000 thru 0xed82d67 (0xed68 bytes) 2024-09-14T07:09:31.241 INFO:teuthology.orchestra.run.smithi145.stdout:3438 write 0x7681a00 thru 0x768ff76 (0xe577 bytes) 2024-09-14T07:09:31.258 INFO:teuthology.orchestra.run.smithi145.stdout:3440 trunc from 0xed82e00 to 0x450be00 2024-09-14T07:09:31.295 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:31.292+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:31.299 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 118.998248543, 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.104255359, 2024-09-14T07:09:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.104135479999997, 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.086910359999997, 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 20.414315973000001, 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.640274107, 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:09:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.640241879000001, 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:09:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:09:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:09:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.640219863999999, 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:09:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.640198929, 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.346 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:09:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.640177918999999, 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:09:31.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:09:31.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.640144219, 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.355 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.640123576000001, 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:09:31.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:09:31.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.639878359000001, 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.363 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.367 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.639846640000002, 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:31.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:09:31.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.639824880999999, 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.372 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:09:31.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:09:31.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.375 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.639792801999999, 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:09:31.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:09:31.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.379 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.638621596, 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:09:31.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:09:31.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:09:31.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.638598633000001, 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.384 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.638577343000001, 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.638557120000002, 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:09:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:09:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:31.396 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:09:31.399 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:31.399 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:31.392+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:31.399 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:31.400 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3441 read 0x3c02000 thru 0x3c02066 (0x67 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3442 read 0x38f2a00 thru 0x38fde43 (0xb444 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3443 punch from 0x3bc800 to 0x3c2000, (0x5800 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3444 write 0x929de00 thru 0x929efe1 (0x11e2 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3446 trunc from 0x929f000 to 0x8689800 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3447 punch from 0x5892000 to 0x589da00, (0xba00 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3449 write 0x9866c00 thru 0x987393f (0xcd40 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3453 write 0x93d8c00 thru 0x93de9b2 (0x5db3 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3454 read 0x801fe00 thru 0x8026e07 (0x7008 bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3455 read 0x2e8c200 thru 0x2e8eaea (0x28eb bytes) 2024-09-14T07:09:31.401 INFO:teuthology.orchestra.run.smithi145.stdout:3456 punch from 0x6157c00 to 0x6159200, (0x1600 bytes) 2024-09-14T07:09:31.402 INFO:teuthology.orchestra.run.smithi145.stdout:3458 read 0x5cca00 thru 0x5d7101 (0xa702 bytes) 2024-09-14T07:09:31.402 INFO:teuthology.orchestra.run.smithi145.stdout:3459 punch from 0x36bfe00 to 0x36cfa00, (0xfc00 bytes) 2024-09-14T07:09:31.402 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:31.420 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:31.421 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:31.423 INFO:teuthology.orchestra.run.smithi145.stdout:3463 write 0xb617600 thru 0xb61f1fd (0x7bfe bytes) 2024-09-14T07:09:31.424 INFO:teuthology.orchestra.run.smithi145.stdout:3467 write 0x9e0ba00 thru 0x9e1afb8 (0xf5b9 bytes) 2024-09-14T07:09:31.424 INFO:teuthology.orchestra.run.smithi145.stdout:3468 punch from 0x6baac00 to 0x6bab200, (0x600 bytes) 2024-09-14T07:09:31.424 INFO:teuthology.orchestra.run.smithi145.stdout:3471 read 0x5cf200 thru 0x5d16bd (0x24be bytes) 2024-09-14T07:09:31.425 INFO:teuthology.orchestra.run.smithi145.stdout:3476 punch from 0x7bad000 to 0x7bbc400, (0xf400 bytes) 2024-09-14T07:09:31.425 INFO:teuthology.orchestra.run.smithi145.stdout:3478 read 0x95b1a00 thru 0x95b65ea (0x4beb bytes) 2024-09-14T07:09:31.425 INFO:teuthology.orchestra.run.smithi145.stdout:3479 write 0x8a98800 thru 0x8aa7398 (0xeb99 bytes) 2024-09-14T07:09:31.441 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:31.441 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:31.449 INFO:teuthology.orchestra.run.smithi145.stdout:3484 write 0xcbbec00 thru 0xcbcabcd (0xbfce bytes) 2024-09-14T07:09:31.450 INFO:teuthology.orchestra.run.smithi145.stdout:3485 write 0x7d9ce00 thru 0x7d9f709 (0x290a bytes) 2024-09-14T07:09:31.450 INFO:teuthology.orchestra.run.smithi145.stdout:3486 write 0x2692200 thru 0x269f201 (0xd002 bytes) 2024-09-14T07:09:31.450 INFO:teuthology.orchestra.run.smithi145.stdout:3487 read 0x99dc200 thru 0x99e33cd (0x71ce bytes) 2024-09-14T07:09:31.450 INFO:teuthology.orchestra.run.smithi145.stdout:3488 write 0x3b74c00 thru 0x3b790dd (0x44de bytes) 2024-09-14T07:09:31.450 INFO:teuthology.orchestra.run.smithi145.stdout:3489 read 0x9245800 thru 0x9247df2 (0x25f3 bytes) 2024-09-14T07:09:31.450 INFO:teuthology.orchestra.run.smithi145.stdout:3490 punch from 0x7d7a800 to 0x7d82400, (0x7c00 bytes) 2024-09-14T07:09:31.451 INFO:teuthology.orchestra.run.smithi145.stdout:3491 punch from 0x99b7800 to 0x99bbe00, (0x4600 bytes) 2024-09-14T07:09:31.461 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:31.474 INFO:teuthology.orchestra.run.smithi145.stdout:3492 trunc from 0xcbcac00 to 0xe67400 2024-09-14T07:09:31.527 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:31.524+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:31.575 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:31.575 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:31.590 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:31.598 INFO:teuthology.orchestra.run.smithi145.stdout:3495 trunc from 0xe67400 to 0x628f800 2024-09-14T07:09:31.599 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:31.592+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:31.610 INFO:teuthology.orchestra.run.smithi145.stdout:3502 read 0x22cfc00 thru 0x22daba2 (0xafa3 bytes) 2024-09-14T07:09:31.625 INFO:teuthology.orchestra.run.smithi145.stdout:3503 write 0x6b57e00 thru 0x6b60698 (0x8899 bytes) 2024-09-14T07:09:31.630 INFO:teuthology.orchestra.run.smithi145.stdout:3504 trunc from 0x6b60800 to 0x8ff3400 2024-09-14T07:09:31.646 INFO:teuthology.orchestra.run.smithi145.stdout:3505 punch from 0x4e1e200 to 0x4e28000, (0x9e00 bytes) 2024-09-14T07:09:31.670 INFO:teuthology.orchestra.run.smithi145.stdout:3506 write 0xe217400 thru 0xe21ae68 (0x3a69 bytes) 2024-09-14T07:09:31.670 INFO:teuthology.orchestra.run.smithi145.stdout:3508 punch from 0x6e93a00 to 0x6e96800, (0x2e00 bytes) 2024-09-14T07:09:31.671 INFO:teuthology.orchestra.run.smithi145.stdout:3509 read 0x9897400 thru 0x989c9f6 (0x55f7 bytes) 2024-09-14T07:09:31.671 INFO:teuthology.orchestra.run.smithi145.stdout:3510 trunc from 0xe21b000 to 0xdecd000 2024-09-14T07:09:31.698 INFO:teuthology.orchestra.run.smithi145.stdout:3513 read 0x436e200 thru 0x436ec21 (0xa22 bytes) 2024-09-14T07:09:31.699 INFO:teuthology.orchestra.run.smithi145.stdout:3517 punch from 0x875ce00 to 0x876a800, (0xda00 bytes) 2024-09-14T07:09:31.699 INFO:teuthology.orchestra.run.smithi145.stdout:3518 write 0x716ee00 thru 0x716fc01 (0xe02 bytes) 2024-09-14T07:09:31.700 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:31.696+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:31.700 INFO:teuthology.orchestra.run.smithi145.stdout:3519 read 0x91fcc00 thru 0x91ffa05 (0x2e06 bytes) 2024-09-14T07:09:31.701 INFO:teuthology.orchestra.run.smithi145.stdout:3520 write 0xd88ae00 thru 0xd8927f6 (0x79f7 bytes) 2024-09-14T07:09:31.701 INFO:teuthology.orchestra.run.smithi145.stdout:3524 write 0xc5c4600 thru 0xc5d40f0 (0xfaf1 bytes) 2024-09-14T07:09:31.701 INFO:teuthology.orchestra.run.smithi145.stdout:3525 write 0x5c96c00 thru 0x5c9879c (0x1b9d bytes) 2024-09-14T07:09:31.701 INFO:teuthology.orchestra.run.smithi145.stdout:3527 read 0xa1e7800 thru 0xa1f506a (0xd86b bytes) 2024-09-14T07:09:31.701 INFO:teuthology.orchestra.run.smithi145.stdout:3528 write 0x6e03400 thru 0x6e0c3f3 (0x8ff4 bytes) 2024-09-14T07:09:31.701 INFO:teuthology.orchestra.run.smithi145.stdout:3530 read 0x9156a00 thru 0x9161a4b (0xb04c bytes) 2024-09-14T07:09:31.702 INFO:teuthology.orchestra.run.smithi145.stdout:3531 read 0x5289000 thru 0x5291261 (0x8262 bytes) 2024-09-14T07:09:31.702 INFO:teuthology.orchestra.run.smithi145.stdout:3532 read 0x905f200 thru 0x90699e8 (0xa7e9 bytes) 2024-09-14T07:09:31.702 INFO:teuthology.orchestra.run.smithi145.stdout:3534 write 0xbcd6400 thru 0xbcd7ddd (0x19de bytes) 2024-09-14T07:09:31.702 INFO:teuthology.orchestra.run.smithi145.stdout:3535 read 0x32f7a00 thru 0x32fa9ac (0x2fad bytes) 2024-09-14T07:09:31.702 INFO:teuthology.orchestra.run.smithi145.stdout:3537 punch from 0x9143200 to 0x914e800, (0xb600 bytes) 2024-09-14T07:09:31.703 INFO:teuthology.orchestra.run.smithi145.stdout:3538 read 0x1458600 thru 0x146073a (0x813b bytes) 2024-09-14T07:09:31.703 INFO:teuthology.orchestra.run.smithi145.stdout:3541 write 0x33bb800 thru 0x33c3443 (0x7c44 bytes) 2024-09-14T07:09:31.703 INFO:teuthology.orchestra.run.smithi145.stdout:3542 clone 25 order 20 su 131072 sc 12 2024-09-14T07:09:31.734 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:31.735 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:31.748 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:31.836 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:31.832+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:31.901 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:31.896+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:31.913 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:31.913 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:31.927 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:32.019 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:32.016+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:32.098 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:32.098 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:32.103 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:32.096+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:32.114 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:32.204 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:32.200+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:32.280 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:32.281 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:32.294 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:32.335 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:32.332+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:32.406 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:32.400+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:32.467 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:32.467 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:32.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:32.495 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone24 from 0xdecd000 (overlap 0x519600) to 0x187800 2024-09-14T07:09:32.508 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:32.500+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:32.608 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:32.604+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:32.627 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:32.627 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:32.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:32.731 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:32.728+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:32.810 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:32.804+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:32.815 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:32.815 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:32.831 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:32.919 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:32.916+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:33.007 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:33.008 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:33.012 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:33.008+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:33.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:33.075 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T07:09:33.113 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:33.108+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:33.203 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:33.203 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:33.213 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:33.208+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:33.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:33.314 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:33.308+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:33.349 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:33.349 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:33.364 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:33.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:09:33.415 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:33.412+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:33.439 INFO:teuthology.orchestra.run.smithi145.stdout:3543 read 0xadd6400 thru 0xade4459 (0xe05a bytes) 2024-09-14T07:09:33.441 INFO:teuthology.orchestra.run.smithi145.stdout:3544 read 0xf6c00 thru 0xf9ff2 (0x33f3 bytes) 2024-09-14T07:09:33.444 INFO:teuthology.orchestra.run.smithi145.stdout:3546 read 0xb8fa000 thru 0xb907929 (0xd92a bytes) 2024-09-14T07:09:33.446 INFO:teuthology.orchestra.run.smithi145.stdout:3549 punch from 0x93b4c00 to 0x93c4600, (0xfa00 bytes) 2024-09-14T07:09:33.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:33.490 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:33.490 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:33.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:33.551 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:33.548+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:33.595 INFO:teuthology.orchestra.run.smithi145.stdout:3550 trunc from 0xdecd000 to 0x902400 2024-09-14T07:09:33.617 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:33.612+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:33.629 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:33.663 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:33.672 INFO:teuthology.orchestra.run.smithi145.stdout:3554 punch from 0x21ee00 to 0x21f200, (0x400 bytes) 2024-09-14T07:09:33.673 INFO:teuthology.orchestra.run.smithi145.stdout:3556 read 0x199000 thru 0x1a3e19 (0xae1a bytes) 2024-09-14T07:09:33.674 INFO:teuthology.orchestra.run.smithi145.stdout:3561 punch from 0x67d400 to 0x686400, (0x9000 bytes) 2024-09-14T07:09:33.681 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:33.688 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:33.688 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:33.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:33.703 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:33.704 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:33.717 INFO:teuthology.orchestra.run.smithi145.stdout:3563 write 0xb660e00 thru 0xb66bbb0 (0xadb1 bytes) 2024-09-14T07:09:33.717 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:33.712+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:33.719 INFO:teuthology.orchestra.run.smithi145.stdout:3564 read 0xcd1000 thru 0xcd4517 (0x3518 bytes) 2024-09-14T07:09:33.719 INFO:teuthology.orchestra.run.smithi145.stdout:3565 write 0x203a200 thru 0x20454eb (0xb2ec bytes) 2024-09-14T07:09:33.719 INFO:teuthology.orchestra.run.smithi145.stdout:3567 write 0x373c200 thru 0x374181f (0x5620 bytes) 2024-09-14T07:09:33.719 INFO:teuthology.orchestra.run.smithi145.stdout:3569 write 0x174c00 thru 0x17dfda (0x93db bytes) 2024-09-14T07:09:33.721 INFO:teuthology.orchestra.run.smithi145.stdout:3570 punch from 0x3225c00 to 0x322bc00, (0x6000 bytes) 2024-09-14T07:09:33.724 INFO:teuthology.orchestra.run.smithi145.stdout:3572 trunc from 0xb66bc00 to 0xa546400 2024-09-14T07:09:33.726 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:33.726 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:33.748 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:33.773 INFO:teuthology.orchestra.run.smithi145.stdout:3573 write 0x52b1c00 thru 0x52bbd16 (0xa117 bytes) 2024-09-14T07:09:33.774 INFO:teuthology.orchestra.run.smithi145.stdout:3580 read 0x13c2000 thru 0x13d07a3 (0xe7a4 bytes) 2024-09-14T07:09:33.775 INFO:teuthology.orchestra.run.smithi145.stdout:3582 read 0x53c0e00 thru 0x53ccde4 (0xbfe5 bytes) 2024-09-14T07:09:33.775 INFO:teuthology.orchestra.run.smithi145.stdout:3590 write 0x1063800 thru 0x1065571 (0x1d72 bytes) 2024-09-14T07:09:33.775 INFO:teuthology.orchestra.run.smithi145.stdout:3593 read 0x5c1c400 thru 0x5c1d979 (0x157a bytes) 2024-09-14T07:09:33.779 INFO:teuthology.orchestra.run.smithi145.stdout:3594 trunc from 0xa546400 to 0xc0fdc00 2024-09-14T07:09:33.812 INFO:teuthology.orchestra.run.smithi145.stdout:3598 punch from 0x77ee200 to 0x77f9600, (0xb400 bytes) 2024-09-14T07:09:33.814 INFO:teuthology.orchestra.run.smithi145.stdout:3603 read 0x8e61800 thru 0x8e65f35 (0x4736 bytes) 2024-09-14T07:09:33.814 INFO:teuthology.orchestra.run.smithi145.stdout:3604 read 0x9797a00 thru 0x97a3072 (0xb673 bytes) 2024-09-14T07:09:33.814 INFO:teuthology.orchestra.run.smithi145.stdout:3606 punch from 0x2cdc000 to 0x2cebe00, (0xfe00 bytes) 2024-09-14T07:09:33.814 INFO:teuthology.orchestra.run.smithi145.stdout:3607 punch from 0x2369a00 to 0x2372a00, (0x9000 bytes) 2024-09-14T07:09:33.818 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:33.812+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:33.828 INFO:teuthology.orchestra.run.smithi145.stdout:3609 trunc from 0xc0fdc00 to 0x72e7a00 2024-09-14T07:09:33.829 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:33.829 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:33.829 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:33.829 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:33.829 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:33.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:33.856 INFO:teuthology.orchestra.run.smithi145.stdout:3611 read 0x682ae00 thru 0x6830c41 (0x5e42 bytes) 2024-09-14T07:09:33.858 INFO:teuthology.orchestra.run.smithi145.stdout:3612 read 0x1c4d200 thru 0x1c5be46 (0xec47 bytes) 2024-09-14T07:09:33.858 INFO:teuthology.orchestra.run.smithi145.stdout:3613 read 0xd7200 thru 0xddc93 (0x6a94 bytes) 2024-09-14T07:09:33.860 INFO:teuthology.orchestra.run.smithi145.stdout:3614 read 0x1d46400 thru 0x1d4da7f (0x7680 bytes) 2024-09-14T07:09:33.865 INFO:teuthology.orchestra.run.smithi145.stdout:3617 trunc from 0x72e7a00 to 0x5481200 2024-09-14T07:09:33.902 INFO:teuthology.orchestra.run.smithi145.stdout:3618 trunc from 0x5481200 to 0x6ee7200 2024-09-14T07:09:33.935 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:33.932+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:33.939 INFO:teuthology.orchestra.run.smithi145.stdout:3619 trunc from 0x6ee7200 to 0xb792000 2024-09-14T07:09:33.969 INFO:teuthology.orchestra.run.smithi145.stdout:3620 write 0x9ec5200 thru 0x9ed0d58 (0xbb59 bytes) 2024-09-14T07:09:33.969 INFO:teuthology.orchestra.run.smithi145.stdout:3622 write 0x38f400 thru 0x39c0af (0xccb0 bytes) 2024-09-14T07:09:33.971 INFO:teuthology.orchestra.run.smithi145.stdout:3625 read 0x9c62c00 thru 0x9c65649 (0x2a4a bytes) 2024-09-14T07:09:33.972 INFO:teuthology.orchestra.run.smithi145.stdout:3626 read 0x9862a00 thru 0x987290f (0xff10 bytes) 2024-09-14T07:09:33.972 INFO:teuthology.orchestra.run.smithi145.stdout:3627 read 0xc18a00 thru 0xc2180b (0x8e0c bytes) 2024-09-14T07:09:33.972 INFO:teuthology.orchestra.run.smithi145.stdout:3628 write 0x1f56000 thru 0x1f63d5b (0xdd5c bytes) 2024-09-14T07:09:33.973 INFO:teuthology.orchestra.run.smithi145.stdout:3629 flatten 2024-09-14T07:09:34.018 INFO:teuthology.orchestra.run.smithi145.stdout:3630 punch from 0x7f95400 to 0x7f96800, (0x1400 bytes) 2024-09-14T07:09:34.019 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:34.016+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:34.020 INFO:teuthology.orchestra.run.smithi145.stdout:3631 read 0x388e400 thru 0x388f510 (0x1111 bytes) 2024-09-14T07:09:34.020 INFO:teuthology.orchestra.run.smithi145.stdout:3632 read 0x2376200 thru 0x237f5ed (0x93ee bytes) 2024-09-14T07:09:34.021 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 76.718448713000001, 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:09:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:09:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 76.673458406999998, 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:09:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:09:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:09:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 72.371525950999995, 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.038 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 72.050198217000002, 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:34.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:09:34.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:09:34.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.613069818, 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:09:34.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:34.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.613059159999999, 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.613044146, 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.613033230999999, 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.613021720999999, 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:09:34.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:09:34.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.613009177999999, 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:09:34.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606173175999999, 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:09:34.070 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:09:34.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.074 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606162582, 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606114801, 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:09:34.079 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:09:34.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:09:34.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606101334000002, 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606090362, 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:09:34.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606079600000001, 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606063546000001, 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:34.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:09:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:09:34.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606046975000002, 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:09:34.100 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:09:34.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606035199000001, 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:09:34.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:34.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.606024521999998, 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.109 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:09:34.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:34.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:09:34.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:09:34.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:34.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:34.113 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:34.114 INFO:teuthology.orchestra.run.smithi145.stdout:3634 read 0x9a15a00 thru 0x9a25602 (0xfc03 bytes) 2024-09-14T07:09:34.114 INFO:teuthology.orchestra.run.smithi145.stdout:3635 read 0x92b4c00 thru 0x92bf44f (0xa850 bytes) 2024-09-14T07:09:34.114 INFO:teuthology.orchestra.run.smithi145.stdout:3638 read 0xb07200 thru 0xb0b9b5 (0x47b6 bytes) 2024-09-14T07:09:34.114 INFO:teuthology.orchestra.run.smithi145.stdout:3639 write 0xde4e200 thru 0xde562c7 (0x80c8 bytes) 2024-09-14T07:09:34.115 INFO:teuthology.orchestra.run.smithi145.stdout:3641 trunc from 0xde56400 to 0xfbbe00 2024-09-14T07:09:34.115 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:34.164 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:34.160+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:34.201 INFO:teuthology.orchestra.run.smithi145.stdout:3642 write 0xac0b600 thru 0xac16f63 (0xb964 bytes) 2024-09-14T07:09:34.202 INFO:teuthology.orchestra.run.smithi145.stdout:3645 read 0xeeb000 thru 0xeedb59 (0x2b5a bytes) 2024-09-14T07:09:34.215 INFO:teuthology.orchestra.run.smithi145.stdout:3646 trunc from 0xac17000 to 0xe1fa200 2024-09-14T07:09:34.221 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:34.216+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:34.254 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.5658.0:868 3.1e e101/25 3:7a62f29a:::rbd_object_map.116170297dfe:head v 101'402, mlcod=101'400)", 2024-09-14T07:09:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:34.253050+0000", 2024-09-14T07:09:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.001377056, 2024-09-14T07:09:34.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013938620000000001, 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253050+0000", 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253050+0000", 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253052+0000", 2024-09-14T07:09:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9800000000000001e-06 2024-09-14T07:09:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253065+0000", 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2947e-05 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253093+0000", 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8011999999999999e-05 2024-09-14T07:09:34.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253113+0000", 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253234+0000", 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012136100000000001 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:34.253271+0000", 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7039000000000002e-05 2024-09-14T07:09:34.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:09:34.259 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:09:34.259 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:34.267 INFO:teuthology.orchestra.run.smithi145.stdout:3647 write 0xe441200 thru 0xe448d65 (0x7b66 bytes) 2024-09-14T07:09:34.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:34.306 INFO:teuthology.orchestra.run.smithi145.stdout:3648 write 0xed1a600 thru 0xed20ec5 (0x68c6 bytes) 2024-09-14T07:09:34.308 INFO:teuthology.orchestra.run.smithi145.stdout:3649 write 0x26e7e00 thru 0x26ed4cd (0x56ce bytes) 2024-09-14T07:09:34.309 INFO:teuthology.orchestra.run.smithi145.stdout:3650 read 0xd401a00 thru 0xd4105d1 (0xebd2 bytes) 2024-09-14T07:09:34.309 INFO:teuthology.orchestra.run.smithi145.stdout:3651 write 0xd116000 thru 0xd12070c (0xa70d bytes) 2024-09-14T07:09:34.309 INFO:teuthology.orchestra.run.smithi145.stdout:3653 write 0x163a600 thru 0x163ff98 (0x5999 bytes) 2024-09-14T07:09:34.322 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:34.320+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:34.322 INFO:teuthology.orchestra.run.smithi145.stdout:3654 trunc from 0xed21000 to 0x84cd600 2024-09-14T07:09:34.345 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:09:34.385 INFO:teuthology.orchestra.run.smithi145.stdout:3656 trunc from 0x84cd600 to 0xaf38c00 2024-09-14T07:09:34.404 INFO:teuthology.orchestra.run.smithi145.stdout:3657 read 0xa34e600 thru 0xa354163 (0x5b64 bytes) 2024-09-14T07:09:34.404 INFO:teuthology.orchestra.run.smithi145.stdout:3660 read 0x912ea00 thru 0x913015a (0x175b bytes) 2024-09-14T07:09:34.405 INFO:teuthology.orchestra.run.smithi145.stdout:3662 write 0x7b71e00 thru 0x7b73926 (0x1b27 bytes) 2024-09-14T07:09:34.405 INFO:teuthology.orchestra.run.smithi145.stdout:3663 read 0x45ee800 thru 0x45f9bcc (0xb3cd bytes) 2024-09-14T07:09:34.405 INFO:teuthology.orchestra.run.smithi145.stdout:3664 read 0xf55800 thru 0xf63f29 (0xe72a bytes) 2024-09-14T07:09:34.405 INFO:teuthology.orchestra.run.smithi145.stdout:3666 read 0x27ff000 thru 0x280695b (0x795c bytes) 2024-09-14T07:09:34.405 INFO:teuthology.orchestra.run.smithi145.stdout:3670 punch from 0x9f7a600 to 0x9f80000, (0x5a00 bytes) 2024-09-14T07:09:34.406 INFO:teuthology.orchestra.run.smithi145.stdout:3673 write 0x861000 thru 0x870bf7 (0xfbf8 bytes) 2024-09-14T07:09:34.422 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:34.420+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:34.434 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:09:34.435 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:34.435 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-14T07:09:34.435 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:09:34.435 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:09:34.435 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-14T07:09:34.436 INFO:teuthology.orchestra.run.smithi145.stdout:3674 write 0xebdfa00 thru 0xebe2f8c (0x358d bytes) 2024-09-14T07:09:34.437 INFO:teuthology.orchestra.run.smithi145.stdout:3676 write 0x6f33600 thru 0x6f40029 (0xca2a bytes) 2024-09-14T07:09:34.437 INFO:teuthology.orchestra.run.smithi145.stdout:3678 clone 26 order 23 su 524288 sc 5 2024-09-14T07:09:34.450 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:34.450 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:34.450 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:34.450 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:34.450 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:34.466 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:34.523 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:34.524+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:34.559 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T07:09:34.559 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:34.624 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:34.624+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:34.628 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:34.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:34.629 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:34.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.629 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:34.629 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:34.629 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 122.326361699, 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:34.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:34.636 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.432368515, 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:34.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:34.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:34.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:34.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.432248635999997, 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:34.646 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:34.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:34.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:34.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:34.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:34.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:34.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:34.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:34.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:34.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:34.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.415023515999998, 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:34.656 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:34.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:34.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:34.662 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.742429129000001, 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:09:34.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:09:34.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.668 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.968387263, 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.669 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.968355034999998, 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.673 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:09:34.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.968333019999999, 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:09:34.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:09:34.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:09:34.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:09:34.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.968312085000001, 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:09:34.683 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:34.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:34.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:09:34.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:09:34.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:09:34.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:09:34.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:09:34.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:09:34.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:09:34.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:09:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:09:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:09:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.690 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.968291075, 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.691 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.692 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:09:34.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.693 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:34.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:09:34.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:09:34.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:09:34.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:09:34.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:09:34.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:09:34.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:09:34.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.699 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.699 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:34.699 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.968257375, 2024-09-14T07:09:34.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:09:34.699 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.700 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:09:34.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.968236732000001, 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:09:34.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:34.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.967991515000001, 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.709 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:34.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:09:34.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:09:34.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.714 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.967959795999999, 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:09:34.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:09:34.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:09:34.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.967938037, 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.719 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:09:34.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.723 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.967905957999999, 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:09:34.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:34.728 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.966734752000001, 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:09:34.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:09:34.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:09:34.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:09:34.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.966711789000001, 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:09:34.734 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:34.738 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.966690498999998, 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.739 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:09:34.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.740 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:34.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:09:34.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:09:34.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:09:34.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:09:34.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:09:34.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:34.746 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 21.966670275999999, 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:34.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:09:34.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:09:34.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:09:34.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:34.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:09:34.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:09:34.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:34.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:34.753 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:34.755 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:34.756 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:34.752+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:34.855 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:34.856+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:34.927 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:34.927 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:34.927 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:34.928 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:34.942 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:34.988 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:34.984+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:35.057 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:35.052+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:35.064 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:35.064 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:35.064 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:35.064 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:35.064 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:35.078 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:35.168 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:35.164+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:35.247 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 77.948599716000004, 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.248 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:35.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.089124128999998, 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.254 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.089021844000001, 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.259 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:35.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:35.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 64.389970003000002, 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:35.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:35.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:35.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.905048811, 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:09:35.268 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:35.272 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.905013590999999, 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:09:35.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.276 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904958442000002, 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.280 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904924373, 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:35.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904881026999998, 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:35.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:09:35.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.288 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904847503999999, 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:35.292 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904800254000001, 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904686005999999, 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904576311, 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904510168000002, 2024-09-14T07:09:35.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904393664000001, 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904328008, 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.311 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:09:35.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:09:35.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:09:35.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904268687999998, 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:35.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.904127819999999, 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.319 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:09:35.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:09:35.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.903712713000001, 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.327 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.903644973999999, 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.328 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:09:35.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:09:35.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.332 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:35.334 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:35.334 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:35.332+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:35.434 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:35.428+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:35.515 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:35.516 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:35.516 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:35.516 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:35.530 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:35.534 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:35.532+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:35.635 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:35.636+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:35.697 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:35.697 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:35.697 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:35.697 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:35.697 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:35.711 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:35.736 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:35.736+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:35.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:35.835 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:35.835 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:35.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 103.002394698, 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:35.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.840 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 103.00162035, 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:35.841 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.845 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.990437352000001, 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:35.846 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:35.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.693509145999997, 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.852 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:35.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:35.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:35.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:35.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.458605322, 2024-09-14T07:09:35.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:09:35.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.458043531000001, 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.862 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.866 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.457548964000001, 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.456931545, 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.871 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.456503286, 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.456107421999999, 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.883 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.455515746, 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.884 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.452186632, 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.889 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:09:35.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:09:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.452145581, 2024-09-14T07:09:35.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:09:35.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:09:35.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.452093321, 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.452054434000001, 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.905 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.451995091000001, 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:09:35.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.451941003000002, 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:35.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.45190165, 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.45186387, 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:09:35.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:35.923 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.451827167000001, 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:35.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:09:35.927 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.928 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:35.928 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:35.928 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:35.929 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:35.930 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:35.928+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:35.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:35.946 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:35.962 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:35.962 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:35.983 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:36.006 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:36.028 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:36.030 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:36.028+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:36.050 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:36.050 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:36.065 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:36.086 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T07:09:36.131 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:36.128+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:36.168 INFO:teuthology.orchestra.run.smithi145.stdout:3681 write 0xd5ac00 thru 0xd6982f (0xec30 bytes) 2024-09-14T07:09:36.170 INFO:teuthology.orchestra.run.smithi145.stdout:3682 read 0xdec6600 thru 0xdecb587 (0x4f88 bytes) 2024-09-14T07:09:36.171 INFO:teuthology.orchestra.run.smithi145.stdout:3683 read 0x3dda200 thru 0x3ddfa43 (0x5844 bytes) 2024-09-14T07:09:36.172 INFO:teuthology.orchestra.run.smithi145.stdout:3687 read 0x9611800 thru 0x962017d (0xe97e bytes) 2024-09-14T07:09:36.173 INFO:teuthology.orchestra.run.smithi145.stdout:3690 punch from 0x233b000 to 0x233ce00, (0x1e00 bytes) 2024-09-14T07:09:36.232 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:36.232+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:36.238 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:36.238 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:36.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:36.323 INFO:teuthology.orchestra.run.smithi145.stdout:3691 trunc from 0xebe3000 to 0x9a3b200 2024-09-14T07:09:36.332 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:36.332+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:36.362 INFO:teuthology.orchestra.run.smithi145.stdout:3692 read 0x345ae00 thru 0x3465980 (0xab81 bytes) 2024-09-14T07:09:36.363 INFO:teuthology.orchestra.run.smithi145.stdout:3693 read 0x3004200 thru 0x300f0b1 (0xaeb2 bytes) 2024-09-14T07:09:36.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:09:36.381 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:36.382 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:36.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:36.399 INFO:teuthology.orchestra.run.smithi145.stdout:3694 write 0xec12c00 thru 0xec22494 (0xf895 bytes) 2024-09-14T07:09:36.401 INFO:teuthology.orchestra.run.smithi145.stdout:3695 punch from 0xb0b2e00 to 0xb0bd400, (0xa600 bytes) 2024-09-14T07:09:36.401 INFO:teuthology.orchestra.run.smithi145.stdout:3696 read 0x8994e00 thru 0x8998577 (0x3778 bytes) 2024-09-14T07:09:36.403 INFO:teuthology.orchestra.run.smithi145.stdout:3698 read 0x9ed1c00 thru 0x9ed7dd1 (0x61d2 bytes) 2024-09-14T07:09:36.409 INFO:teuthology.orchestra.run.smithi145.stdout:3700 trunc from 0xec22600 to 0xd553600 2024-09-14T07:09:36.433 INFO:teuthology.orchestra.run.smithi145.stdout:3704 punch from 0x55d3200 to 0x55d7800, (0x4600 bytes) 2024-09-14T07:09:36.434 INFO:teuthology.orchestra.run.smithi145.stdout:3707 read 0x8202000 thru 0x8204fef (0x2ff0 bytes) 2024-09-14T07:09:36.437 INFO:teuthology.orchestra.run.smithi145.stdout:3709 read 0x799b600 thru 0x79a9948 (0xe349 bytes) 2024-09-14T07:09:36.439 INFO:teuthology.orchestra.run.smithi145.stdout:3710 trunc from 0xd553600 to 0xd808a00 2024-09-14T07:09:36.447 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:36.448+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:36.458 INFO:teuthology.orchestra.run.smithi145.stdout:3711 write 0x315f000 thru 0x3162098 (0x3099 bytes) 2024-09-14T07:09:36.459 INFO:teuthology.orchestra.run.smithi145.stdout:3712 punch from 0x14f8a00 to 0x14fb000, (0x2600 bytes) 2024-09-14T07:09:36.460 INFO:teuthology.orchestra.run.smithi145.stdout:3713 punch from 0xc4c6000 to 0xc4c8000, (0x2000 bytes) 2024-09-14T07:09:36.490 INFO:teuthology.orchestra.run.smithi145.stdout:3714 write 0xea51400 thru 0xea590d3 (0x7cd4 bytes) 2024-09-14T07:09:36.492 INFO:teuthology.orchestra.run.smithi145.stdout:3717 write 0xc0dc00 thru 0xc1c2d1 (0xe6d2 bytes) 2024-09-14T07:09:36.492 INFO:teuthology.orchestra.run.smithi145.stdout:3720 read 0xb153a00 thru 0xb153acf (0xd0 bytes) 2024-09-14T07:09:36.493 INFO:teuthology.orchestra.run.smithi145.stdout:3722 punch from 0x69f5800 to 0x69fcc00, (0x7400 bytes) 2024-09-14T07:09:36.494 INFO:teuthology.orchestra.run.smithi145.stdout:3724 write 0xc6ea200 thru 0xc6ea472 (0x273 bytes) 2024-09-14T07:09:36.494 INFO:teuthology.orchestra.run.smithi145.stdout:3725 write 0x1815e00 thru 0x181b97b (0x5b7c bytes) 2024-09-14T07:09:36.532 INFO:teuthology.orchestra.run.smithi145.stdout:3726 trunc from 0xea59200 to 0x7fc6c00 2024-09-14T07:09:36.534 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:36.532+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:36.576 INFO:teuthology.orchestra.run.smithi145.stdout:3727 read 0xfab200 thru 0xfb96be (0xe4bf bytes) 2024-09-14T07:09:36.599 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:36.599 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:36.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:36.618 INFO:teuthology.orchestra.run.smithi145.stdout:3728 write 0xe73e800 thru 0xe743449 (0x4c4a bytes) 2024-09-14T07:09:36.620 INFO:teuthology.orchestra.run.smithi145.stdout:3730 write 0x58b3e00 thru 0x58c376b (0xf96c bytes) 2024-09-14T07:09:36.630 INFO:teuthology.orchestra.run.smithi145.stdout:3731 trunc from 0xe743600 to 0x9e2e800 2024-09-14T07:09:36.659 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:36.660+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:36.720 INFO:teuthology.orchestra.run.smithi145.stdout:3732 write 0xd8e3800 thru 0xd8ebda5 (0x85a6 bytes) 2024-09-14T07:09:36.722 INFO:teuthology.orchestra.run.smithi145.stdout:3733 read 0x64c6200 thru 0x64cb9b5 (0x57b6 bytes) 2024-09-14T07:09:36.723 INFO:teuthology.orchestra.run.smithi145.stdout:3734 read 0x5119e00 thru 0x511dd0c (0x3f0d bytes) 2024-09-14T07:09:36.724 INFO:teuthology.orchestra.run.smithi145.stdout:3735 write 0x1cfe00 thru 0x1d69b7 (0x6bb8 bytes) 2024-09-14T07:09:36.736 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:36.736+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:36.756 INFO:teuthology.orchestra.run.smithi145.stdout:3736 write 0xe1e3200 thru 0xe1e72c7 (0x40c8 bytes) 2024-09-14T07:09:36.759 INFO:teuthology.orchestra.run.smithi145.stdout:3737 write 0xd64a800 thru 0xd6555ba (0xadbb bytes) 2024-09-14T07:09:36.761 INFO:teuthology.orchestra.run.smithi145.stdout:3740 read 0x2ccaa00 thru 0x2cd8f6f (0xe570 bytes) 2024-09-14T07:09:36.762 INFO:teuthology.orchestra.run.smithi145.stdout:3741 read 0x3e80600 thru 0x3e8eb48 (0xe549 bytes) 2024-09-14T07:09:36.762 INFO:teuthology.orchestra.run.smithi145.stdout:3746 read 0x34cd800 thru 0x34dd615 (0xfe16 bytes) 2024-09-14T07:09:36.763 INFO:teuthology.orchestra.run.smithi145.stdout:3748 write 0xdfa800 thru 0xe04287 (0x9a88 bytes) 2024-09-14T07:09:36.764 INFO:teuthology.orchestra.run.smithi145.stdout:3749 write 0xad78800 thru 0xad83da4 (0xb5a5 bytes) 2024-09-14T07:09:36.765 INFO:teuthology.orchestra.run.smithi145.stdout:3751 write 0x5628e00 thru 0x562c30c (0x350d bytes) 2024-09-14T07:09:36.767 INFO:teuthology.orchestra.run.smithi145.stdout:3753 write 0x46e800 thru 0x47df29 (0xf72a bytes) 2024-09-14T07:09:36.769 INFO:teuthology.orchestra.run.smithi145.stdout:3755 write 0x9276800 thru 0x9278dc4 (0x25c5 bytes) 2024-09-14T07:09:36.772 INFO:teuthology.orchestra.run.smithi145.stdout:3756 write 0x105e600 thru 0x1066daa (0x87ab bytes) 2024-09-14T07:09:36.780 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:36.780 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:36.783 INFO:teuthology.orchestra.run.smithi145.stdout:3758 trunc from 0xe1e7400 to 0x9ac4200 2024-09-14T07:09:36.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:36.843 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:36.844+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:36.868 INFO:teuthology.orchestra.run.smithi145.stdout:3759 write 0x92f2200 thru 0x92f4778 (0x2579 bytes) 2024-09-14T07:09:36.874 INFO:teuthology.orchestra.run.smithi145.stdout:3760 write 0x83d3a00 thru 0x83e191d (0xdf1e bytes) 2024-09-14T07:09:36.877 INFO:teuthology.orchestra.run.smithi145.stdout:3763 read 0x9a0de00 thru 0x9a1b4da (0xd6db bytes) 2024-09-14T07:09:36.879 INFO:teuthology.orchestra.run.smithi145.stdout:3764 write 0x3a79000 thru 0x3a7976c (0x76d bytes) 2024-09-14T07:09:36.879 INFO:teuthology.orchestra.run.smithi145.stdout:3766 punch from 0x4174000 to 0x417d400, (0x9400 bytes) 2024-09-14T07:09:36.888 INFO:teuthology.orchestra.run.smithi145.stdout:3767 trunc from 0x9ac4200 to 0x77e7200 2024-09-14T07:09:36.929 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:36.929 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:36.937 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:36.936+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:36.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:36.985 INFO:teuthology.orchestra.run.smithi145.stdout:3768 trunc from 0x77e7200 to 0x340e600 2024-09-14T07:09:37.029 INFO:teuthology.orchestra.run.smithi145.stdout:3769 trunc from 0x340e600 to 0x5257e00 2024-09-14T07:09:37.038 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:37.036+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:37.080 INFO:teuthology.orchestra.run.smithi145.stdout:3770 write 0x8124600 thru 0x81272e0 (0x2ce1 bytes) 2024-09-14T07:09:37.120 INFO:teuthology.orchestra.run.smithi145.stdout:3771 write 0xcb08400 thru 0xcb11c26 (0x9827 bytes) 2024-09-14T07:09:37.124 INFO:teuthology.orchestra.run.smithi145.stdout:3773 write 0xc8b9600 thru 0xc8ba95f (0x1360 bytes) 2024-09-14T07:09:37.124 INFO:teuthology.orchestra.run.smithi145.stdout:3774 read 0x63be000 thru 0x63c304a (0x504b bytes) 2024-09-14T07:09:37.124 INFO:teuthology.orchestra.run.smithi145.stdout:3775 write 0x49ba800 thru 0x49c3bcb (0x93cc bytes) 2024-09-14T07:09:37.125 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:37.125 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:37.125 INFO:teuthology.orchestra.run.smithi145.stdout:3776 punch from 0x7c1ca00 to 0x7c28000, (0xb600 bytes) 2024-09-14T07:09:37.126 INFO:teuthology.orchestra.run.smithi145.stdout:3779 write 0x242e200 thru 0x242f683 (0x1484 bytes) 2024-09-14T07:09:37.128 INFO:teuthology.orchestra.run.smithi145.stdout:3785 write 0x4893c00 thru 0x4894332 (0x733 bytes) 2024-09-14T07:09:37.139 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:37.136+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:37.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:37.162 INFO:teuthology.orchestra.run.smithi145.stdout:3786 write 0xe188600 thru 0xe192883 (0xa284 bytes) 2024-09-14T07:09:37.166 INFO:teuthology.orchestra.run.smithi145.stdout:3787 read 0x705fe00 thru 0x706d8b9 (0xdaba bytes) 2024-09-14T07:09:37.166 INFO:teuthology.orchestra.run.smithi145.stdout:3792 punch from 0x918d800 to 0x9191800, (0x4000 bytes) 2024-09-14T07:09:37.167 INFO:teuthology.orchestra.run.smithi145.stdout:3793 write 0xf03000 thru 0xf0c328 (0x9329 bytes) 2024-09-14T07:09:37.168 INFO:teuthology.orchestra.run.smithi145.stdout:3794 read 0xc65a400 thru 0xc65ee28 (0x4a29 bytes) 2024-09-14T07:09:37.183 INFO:teuthology.orchestra.run.smithi145.stdout:3796 trunc from 0xe192a00 to 0x5f76e00 2024-09-14T07:09:37.220 INFO:teuthology.orchestra.run.smithi145.stdout:3797 read 0x566fc00 thru 0x5679a4d (0x9e4e bytes) 2024-09-14T07:09:37.223 INFO:teuthology.orchestra.run.smithi145.stdout:3798 trunc from 0x5f76e00 to 0x6c48800 2024-09-14T07:09:37.239 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:37.240+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:37.248 INFO:teuthology.orchestra.run.smithi145.stdout:3799 flatten 2024-09-14T07:09:37.273 INFO:teuthology.orchestra.run.smithi145.stdout:3800 punch from 0x693fe00 to 0x694c200, (0xc400 bytes) 2024-09-14T07:09:37.305 INFO:teuthology.orchestra.run.smithi145.stdout:3801 write 0x863ea00 thru 0x8644649 (0x5c4a bytes) 2024-09-14T07:09:37.308 INFO:teuthology.orchestra.run.smithi145.stdout:3802 read 0x3d90c00 thru 0x3d9f792 (0xeb93 bytes) 2024-09-14T07:09:37.309 INFO:teuthology.orchestra.run.smithi145.stdout:3803 read 0x4ea7800 thru 0x4eb5eb1 (0xe6b2 bytes) 2024-09-14T07:09:37.310 INFO:teuthology.orchestra.run.smithi145.stdout:3804 punch from 0x63aa000 to 0x63aea00, (0x4a00 bytes) 2024-09-14T07:09:37.311 INFO:teuthology.orchestra.run.smithi145.stdout:3805 read 0x2904200 thru 0x2906d6d (0x2b6e bytes) 2024-09-14T07:09:37.316 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:37.316 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:37.323 INFO:teuthology.orchestra.run.smithi145.stdout:3806 trunc from 0x8644800 to 0xc9e3800 2024-09-14T07:09:37.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:37.349 INFO:teuthology.orchestra.run.smithi145.stdout:3807 punch from 0x2ec8600 to 0x2ed5800, (0xd200 bytes) 2024-09-14T07:09:37.352 INFO:teuthology.orchestra.run.smithi145.stdout:3809 punch from 0x2af4000 to 0x2af7600, (0x3600 bytes) 2024-09-14T07:09:37.352 INFO:teuthology.orchestra.run.smithi145.stdout:3810 read 0x6fb5600 thru 0x6fbbac6 (0x64c7 bytes) 2024-09-14T07:09:37.352 INFO:teuthology.orchestra.run.smithi145.stdout:3811 read 0x690c600 thru 0x691641f (0x9e20 bytes) 2024-09-14T07:09:37.354 INFO:teuthology.orchestra.run.smithi145.stdout:3812 write 0xe89400 thru 0xe9096c (0x756d bytes) 2024-09-14T07:09:37.356 INFO:teuthology.orchestra.run.smithi145.stdout:3814 read 0xb7c4200 thru 0xb7c9b33 (0x5934 bytes) 2024-09-14T07:09:37.356 INFO:teuthology.orchestra.run.smithi145.stdout:3815 read 0x69aaa00 thru 0x69b92a6 (0xe8a7 bytes) 2024-09-14T07:09:37.357 INFO:teuthology.orchestra.run.smithi145.stdout:3816 read 0xb452600 thru 0xb45ec57 (0xc658 bytes) 2024-09-14T07:09:37.357 INFO:teuthology.orchestra.run.smithi145.stdout:3817 read 0x1c77400 thru 0x1c792c9 (0x1eca bytes) 2024-09-14T07:09:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:09:37.368 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:37.368+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:37.441 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:37.440+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:37.492 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:37.492 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:37.506 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:37.552 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:37.552+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:37.639 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:37.640 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:37.642 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:37.640+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:37.647 INFO:teuthology.orchestra.run.smithi145.stdout:3818 write 0xe12ec00 thru 0xe132da5 (0x41a6 bytes) 2024-09-14T07:09:37.648 INFO:teuthology.orchestra.run.smithi145.stdout:3820 trunc from 0xe132e00 to 0xe343800 2024-09-14T07:09:37.654 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:37.674 INFO:teuthology.orchestra.run.smithi145.stdout:3824 punch from 0xba02200 to 0xba07000, (0x4e00 bytes) 2024-09-14T07:09:37.675 INFO:teuthology.orchestra.run.smithi145.stdout:3828 punch from 0x32c5800 to 0x32d3e00, (0xe600 bytes) 2024-09-14T07:09:37.676 INFO:teuthology.orchestra.run.smithi145.stdout:3830 write 0x511ca00 thru 0x5124852 (0x7e53 bytes) 2024-09-14T07:09:37.676 INFO:teuthology.orchestra.run.smithi145.stdout:3833 read 0x1173a00 thru 0x1183236 (0xf837 bytes) 2024-09-14T07:09:37.677 INFO:teuthology.orchestra.run.smithi145.stdout:3836 read 0x1f03600 thru 0x1f0b060 (0x7a61 bytes) 2024-09-14T07:09:37.689 INFO:teuthology.orchestra.run.smithi145.stdout:3837 trunc from 0xe343800 to 0xa6a0a00 2024-09-14T07:09:37.732 INFO:teuthology.orchestra.run.smithi145.stdout:3838 write 0x7ce2e00 thru 0x7ce642c (0x362d bytes) 2024-09-14T07:09:37.734 INFO:teuthology.orchestra.run.smithi145.stdout:3840 read 0xf1d000 thru 0xf289e3 (0xb9e4 bytes) 2024-09-14T07:09:37.734 INFO:teuthology.orchestra.run.smithi145.stdout:3842 read 0x85db200 thru 0x85e95bb (0xe3bc bytes) 2024-09-14T07:09:37.735 INFO:teuthology.orchestra.run.smithi145.stdout:3844 punch from 0xa500c00 to 0xa504a00, (0x3e00 bytes) 2024-09-14T07:09:37.736 INFO:teuthology.orchestra.run.smithi145.stdout:3845 write 0x4cec400 thru 0x4cfa858 (0xe459 bytes) 2024-09-14T07:09:37.737 INFO:teuthology.orchestra.run.smithi145.stdout:3846 punch from 0x53e4c00 to 0x53ea200, (0x5600 bytes) 2024-09-14T07:09:37.738 INFO:teuthology.orchestra.run.smithi145.stdout:3848 write 0xa62a800 thru 0xa632e23 (0x8624 bytes) 2024-09-14T07:09:37.739 INFO:teuthology.orchestra.run.smithi145.stdout:3849 write 0xa12aa00 thru 0xa1343c8 (0x99c9 bytes) 2024-09-14T07:09:37.741 INFO:teuthology.orchestra.run.smithi145.stdout:3850 punch from 0x6527600 to 0x652e400, (0x6e00 bytes) 2024-09-14T07:09:37.742 INFO:teuthology.orchestra.run.smithi145.stdout:3852 read 0x46f6200 thru 0x46f9f67 (0x3d68 bytes) 2024-09-14T07:09:37.743 INFO:teuthology.orchestra.run.smithi145.stdout:3853 read 0x35a2e00 thru 0x35ad3e2 (0xa5e3 bytes) 2024-09-14T07:09:37.743 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:37.740+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:37.744 INFO:teuthology.orchestra.run.smithi145.stdout:3855 write 0x4562600 thru 0x4569f2a (0x792b bytes) 2024-09-14T07:09:37.751 INFO:teuthology.orchestra.run.smithi145.stdout:3856 read 0x830ec00 thru 0x8318bf2 (0x9ff3 bytes) 2024-09-14T07:09:37.752 INFO:teuthology.orchestra.run.smithi145.stdout:3858 write 0x1e40000 thru 0x1e4ac63 (0xac64 bytes) 2024-09-14T07:09:37.753 INFO:teuthology.orchestra.run.smithi145.stdout:3859 punch from 0x8865400 to 0x8867800, (0x2400 bytes) 2024-09-14T07:09:37.753 INFO:teuthology.orchestra.run.smithi145.stdout:3861 trunc from 0xa6a0a00 to 0xa76b200 2024-09-14T07:09:37.782 INFO:teuthology.orchestra.run.smithi145.stdout:3863 clone 27 order 18 su 1024 sc 14 2024-09-14T07:09:37.826 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:37.826 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:37.840 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:37.843 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:37.840+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:37.944 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:37.944+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:37.956 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:37.956 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:37.970 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:38.044 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:38.044+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:38.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:38.087 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:38.088 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:38.102 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:38.148 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:38.148+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:38.203 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:38.203 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:38.220 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:38.224 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:38.224 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:38.237 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:38.239 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:38.246 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:38.244+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:38.254 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:38.255 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:38.275 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:38.275 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:38.295 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:38.355 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:38.356+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:38.406 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:38.406 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:38.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:38.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:38.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.406 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:38.406 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.104998824000006, 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:38.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:38.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.245523237, 2024-09-14T07:09:38.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:38.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.245420952000003, 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.417 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:38.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:38.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.546369111000004, 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061447919000003, 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:38.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:09:38.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061412699000002, 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.430 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.434 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061357549999997, 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:09:38.438 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061323481000002, 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:09:38.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:09:38.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061280134999997, 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.443 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061246611999998, 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.447 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:09:38.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061199362000004, 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:09:38.451 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:09:38.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:09:38.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.061085114000001, 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:09:38.455 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060975419000002, 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.459 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:09:38.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060909275999997, 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:09:38.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060792771999999, 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:38.470 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060727116000002, 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:09:38.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:38.474 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060667795999997, 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:09:38.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:09:38.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060526928000002, 2024-09-14T07:09:38.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:09:38.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:09:38.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060111821, 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:09:38.482 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.060044081999997, 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.486 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:09:38.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:09:38.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:09:38.490 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.490 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.490 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.490 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.490 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.490 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:38.491 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:38.492 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:38.492+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:38.593 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:38.592+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:38.619 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:38.619 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:38.620 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:38.620 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:38.633 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:38.693 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:38.692+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:38.758 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:38.759 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:38.759 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:38.759 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:38.759 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:38.773 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:38.794 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:38.792+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:38.896 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:38.896+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:38.954 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:38.954 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.121681999, 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:38.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:38.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:38.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.120907651, 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 77.109724653000001, 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.965 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.970 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.812796446999997, 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:38.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:38.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.577892622999997, 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.979 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:09:38.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.577330832000001, 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:38.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:38.987 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.576836264999997, 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:38.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.991 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.576218846000003, 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:09:38.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:09:38.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.575790587, 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:09:38.996 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:09:38.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:09:38.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.000 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.575394723000002, 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:09:39.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.004 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.574803047000003, 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.005 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:09:39.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571473933, 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:09:39.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:09:39.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571432882000003, 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:09:39.013 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:09:39.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571380622, 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:09:39.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:09:39.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:09:39.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571341734999997, 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:09:39.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:09:39.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:09:39.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:09:39.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571282392000001, 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:09:39.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:09:39.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:39.030 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571228304000002, 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:39.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.034 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571188951000003, 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.035 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:09:39.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571151170999997, 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.039 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:09:39.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:09:39.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.571114467999998, 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:09:39.043 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:39.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:09:39.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:39.047 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:39.048 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T07:09:39.049 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:39.048+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:39.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:39.150 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:39.148+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:39.182 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:39.182 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:39.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:39.250 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:39.247+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:39.332 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:39.333 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:39.348 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:39.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:09:39.396 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:39.395+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:39.452 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:39.451+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:39.530 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.239610932000005, 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:39.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.210103301000004, 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.201455941999996, 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:39.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.535363602000004, 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.368634376000003, 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:39.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:39.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.435633367999998, 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:39.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1413 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680244646999999, 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039824520000000002, 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1413 2024-09-14T07:09:39.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-07 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848109+0000", 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848113+0000", 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6079999999999998e-06 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848116+0000", 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887648+0000", 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039532253000000003 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887688+0000", 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9777000000000002e-05 2024-09-14T07:09:39.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887932+0000", 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024362200000000001 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1414 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680232228000001, 2024-09-14T07:09:39.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040080681999999999, 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1414 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848121+0000", 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.122e-06 2024-09-14T07:09:39.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848124+0000", 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.994e-06 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848126+0000", 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5100000000000001e-06 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887949+0000", 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039822716000000001 2024-09-14T07:09:39.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887988+0000", 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8862999999999997e-05 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888200+0000", 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021226400000000001 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.568 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1415 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680215617000002, 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040391931999999998, 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1415 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848137+0000", 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1e-07 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848139+0000", 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.052e-06 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848146+0000", 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3849999999999996e-06 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848150+0000", 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5420000000000001e-06 2024-09-14T07:09:39.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888232+0000", 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040082568999999998 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888274+0000", 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1869000000000002e-05 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888528+0000", 2024-09-14T07:09:39.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025420500000000002 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680190196000002, 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.577 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680172274, 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:09:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680159504999999, 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:09:39.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680149328999999, 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:09:39.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.591 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680139134000001, 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:39.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.680129114, 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:09:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:09:39.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1642 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 831488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.626718260000001, 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039801272999999998, 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1642 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901637+0000", 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4959999999999999e-06 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901643+0000", 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4710000000000004e-06 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901648+0000", 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941196+0000", 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039548176999999997 2024-09-14T07:09:39.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941234+0000", 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7966999999999997e-05 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941435+0000", 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020081799999999999 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1643 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 854016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.626696279000001, 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040041757999999997, 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1643 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901658+0000", 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3190000000000002e-06 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901666+0000", 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3769999999999998e-06 2024-09-14T07:09:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901671+0000", 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.006e-06 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941457+0000", 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039786119000000002 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941496+0000", 2024-09-14T07:09:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8742000000000003e-05 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941697+0000", 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201795 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1644 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.626673921999998, 2024-09-14T07:09:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040289206000000001, 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1644 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901681+0000", 2024-09-14T07:09:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2170000000000001e-06 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901687+0000", 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8009999999999998e-06 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901693+0000", 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941720+0000", 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040027342 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941758+0000", 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8095e-05 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941967+0000", 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000209139 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1645 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.626651588000001, 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040527821999999998, 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1645 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901701+0000", 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901703+0000", 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.176e-06 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901711+0000", 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1519999999999992e-06 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901716+0000", 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0810000000000001e-06 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941990+0000", 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040274094000000003 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942027+0000", 2024-09-14T07:09:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6726e-05 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942228+0000", 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020122599999999999 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.626628, 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:09:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:09:39.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:09:39.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:39.624 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:39.624 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:39.624 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:09:39.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T07:09:39.626 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:39.626 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:39.623+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:39.726 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:39.723+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:39.809 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:39.809 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:39.809 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:39.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:39.867 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:39.867+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:39.928 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:39.928+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:39.989 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:39.989 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:39.989 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:40.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:40.047 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:40.047+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:40.130 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:40.128+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:40.131 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.131 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.449095103000005, 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.058119265999999, 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:09:40.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:09:40.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.058089467999999, 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.058057193000003, 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:09:40.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:09:40.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057999090000003, 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.148 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:09:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:09:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057958188000001, 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:09:40.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057896380000003, 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:09:40.156 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:09:40.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:09:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057862063999998, 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:40.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:09:40.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057843392999999, 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:09:40.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057819357, 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:09:40.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057799322999998, 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:09:40.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057784488000003, 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:40.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:09:40.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:09:40.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:09:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:09:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057769862000001, 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:09:40.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057755360999998, 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:09:40.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:09:40.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057737606000003, 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:09:40.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057722712, 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057708601000002, 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057686672000003, 2024-09-14T07:09:40.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:09:40.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:40.208 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 33.057665360999998, 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:40.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:09:40.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.770314460000002, 2024-09-14T07:09:40.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:09:40.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:09:40.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:40.217 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:40.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:40.231 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:40.228+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:40.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:40.359 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:40.359+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:40.391 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T07:09:40.394 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:09:40.395 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:40.395 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:40.395 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:40.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T07:09:40.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:40.467 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:40.468 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:40.467+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:40.484 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:40.484 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:40.502 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:40.503 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:40.524 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:40.524 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:40.532 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:40.531+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:40.545 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:40.545 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:40.566 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:40.586 INFO:teuthology.orchestra.run.smithi145.stdout:3866 read 0x31e5000 thru 0x31f3941 (0xe942 bytes) 2024-09-14T07:09:40.598 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:40.598 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:40.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:40.614 INFO:teuthology.orchestra.run.smithi145.stdout:3867 read 0x3660c00 thru 0x366f82d (0xec2e bytes) 2024-09-14T07:09:40.622 INFO:teuthology.orchestra.run.smithi145.stdout:3868 punch from 0x172f800 to 0x1737600, (0x7e00 bytes) 2024-09-14T07:09:40.634 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:40.632+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:40.734 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:40.732+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:40.771 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:40.771 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:40.786 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:40.834 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:40.832+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:40.936 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:40.935+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:40.947 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:40.947 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:40.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:40.973 INFO:teuthology.orchestra.run.smithi145.stdout:3871 read 0x16b8000 thru 0x16c5efc (0xdefd bytes) 2024-09-14T07:09:40.975 INFO:teuthology.orchestra.run.smithi145.stdout:3872 punch from 0x566c800 to 0x566ee00, (0x2600 bytes) 2024-09-14T07:09:41.059 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:41.060+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:41.136 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:41.136 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:41.137 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:41.135+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:41.150 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:41.156 INFO:teuthology.orchestra.run.smithi145.stdout:3874 trunc from 0xa76b200 to 0xd4bc400 2024-09-14T07:09:41.174 INFO:teuthology.orchestra.run.smithi145.stdout:3875 punch from 0x749800 to 0x74c600, (0x2e00 bytes) 2024-09-14T07:09:41.238 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:41.236+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:41.334 INFO:teuthology.orchestra.run.smithi145.stdout:3876 write 0xc39b400 thru 0xc3a07d1 (0x53d2 bytes) 2024-09-14T07:09:41.334 INFO:teuthology.orchestra.run.smithi145.stdout:3878 punch from 0x26d9e00 to 0x26e8600, (0xe800 bytes) 2024-09-14T07:09:41.338 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:41.336+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:41.378 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:41.378 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:41.401 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T07:09:41.439 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:41.439+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:41.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T07:09:41.494 INFO:teuthology.orchestra.run.smithi145.stdout:3879 punch from 0xb0f2c00 to 0xb100400, (0xd800 bytes) 2024-09-14T07:09:41.496 INFO:teuthology.orchestra.run.smithi145.stdout:3880 write 0x2639a00 thru 0x26469d2 (0xcfd3 bytes) 2024-09-14T07:09:41.543 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:41.543+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:41.584 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:41.584 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:41.598 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:41.635 INFO:teuthology.orchestra.run.smithi145.stdout:3881 read 0xafa0800 thru 0xafaf1e0 (0xe9e1 bytes) 2024-09-14T07:09:41.639 INFO:teuthology.orchestra.run.smithi145.stdout:3883 write 0xcf80400 thru 0xcf80a70 (0x671 bytes) 2024-09-14T07:09:41.639 INFO:teuthology.orchestra.run.smithi145.stdout:3884 write 0x685aa00 thru 0x68638b4 (0x8eb5 bytes) 2024-09-14T07:09:41.639 INFO:teuthology.orchestra.run.smithi145.stdout:3886 read 0xc7aa800 thru 0xc7affef (0x57f0 bytes) 2024-09-14T07:09:41.646 INFO:teuthology.orchestra.run.smithi145.stdout:3887 trunc from 0xd4bc400 to 0xdb9ea00 2024-09-14T07:09:41.647 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:41.648+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:41.741 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:41.739+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:41.768 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:41.768 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:41.785 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:41.842 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:41.839+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:41.943 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:41.940+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:41.982 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:41.983 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:41.989 INFO:teuthology.orchestra.run.smithi145.stdout:3888 punch from 0x39c3a00 to 0x39c7600, (0x3c00 bytes) 2024-09-14T07:09:41.998 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:42.044 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:42.043+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:42.145 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:42.144+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:42.157 INFO:teuthology.orchestra.run.smithi145.stdout:3889 punch from 0x4997e00 to 0x49a7200, (0xf400 bytes) 2024-09-14T07:09:42.168 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:42.168 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:42.182 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:42.246 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:42.243+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:42.346 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:42.343+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:42.359 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:42.359 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:42.374 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:42.409 INFO:teuthology.orchestra.run.smithi145.stdout:3893 write 0x61b7e00 thru 0x61c133b (0x953c bytes) 2024-09-14T07:09:42.410 INFO:teuthology.orchestra.run.smithi145.stdout:3895 read 0x5982600 thru 0x598572e (0x312f bytes) 2024-09-14T07:09:42.414 INFO:teuthology.orchestra.run.smithi145.stdout:3896 punch from 0x56c0200 to 0x56cc800, (0xc600 bytes) 2024-09-14T07:09:42.428 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:09:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T07:09:42.447 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:42.448+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:42.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T07:09:42.525 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:42.526 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:42.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:42.587 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:42.588+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:42.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:42.618 INFO:teuthology.orchestra.run.smithi145.stdout:3897 write 0xdb6c200 thru 0xdb7af13 (0xed14 bytes) 2024-09-14T07:09:42.621 INFO:teuthology.orchestra.run.smithi145.stdout:3900 trunc from 0xdb9ea00 to 0xe0e9c00 2024-09-14T07:09:42.649 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:42.648+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:42.750 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:42.748+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:42.755 INFO:teuthology.orchestra.run.smithi145.stdout:3902 punch from 0x712c200 to 0x7132a00, (0x6800 bytes) 2024-09-14T07:09:42.801 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:42.801 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:42.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:42.867 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:42.867+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:42.884 INFO:teuthology.orchestra.run.smithi145.stdout:3904 punch from 0x2bd5400 to 0x2be4200, (0xee00 bytes) 2024-09-14T07:09:42.903 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:42.954 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:42.951+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:42.956 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:42.956 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:42.998 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:43.005 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:43.005 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:43.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:43.038 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:43.057 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:43.056+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:43.062 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:43.081 INFO:teuthology.orchestra.run.smithi145.stdout:3906 read 0xc37c000 thru 0xc380150 (0x4151 bytes) 2024-09-14T07:09:43.088 INFO:teuthology.orchestra.run.smithi145.stdout:3907 read 0x7ccd800 thru 0x7cdc1ef (0xe9f0 bytes) 2024-09-14T07:09:43.092 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:43.092 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:43.106 INFO:teuthology.orchestra.run.smithi145.stdout:3908 read 0xae38800 thru 0xae419b7 (0x91b8 bytes) 2024-09-14T07:09:43.157 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:43.155+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:43.164 INFO:teuthology.orchestra.run.smithi145.stdout:3911 trunc from 0xe0e9c00 to 0x73bb800 2024-09-14T07:09:43.165 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:43.165 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:43.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 85.875251410000004, 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.166 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:43.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:43.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:43.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:43.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 85.845743779000003, 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:43.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.175 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 85.837096419999995, 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 81.171004080000003, 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:43.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:43.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 81.004274854000002, 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:43.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:43.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.071273845999997, 2024-09-14T07:09:43.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1413 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:43.193 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315885125000001, 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039824520000000002, 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1413 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:43.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848107+0000", 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-07 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848109+0000", 2024-09-14T07:09:43.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848113+0000", 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6079999999999998e-06 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848116+0000", 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887648+0000", 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039532253000000003 2024-09-14T07:09:43.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887688+0000", 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9777000000000002e-05 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887932+0000", 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024362200000000001 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.197 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1414 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315872706, 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040080681999999999, 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1414 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848120+0000", 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848121+0000", 2024-09-14T07:09:43.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.122e-06 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848124+0000", 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.994e-06 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848126+0000", 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5100000000000001e-06 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887949+0000", 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039822716000000001 2024-09-14T07:09:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.887988+0000", 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8862999999999997e-05 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888200+0000", 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021226400000000001 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1415 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 448512~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315856095000001, 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040391931999999998, 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1415 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848136+0000", 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848137+0000", 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1e-07 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848139+0000", 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.052e-06 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848146+0000", 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3849999999999996e-06 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848150+0000", 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5420000000000001e-06 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888232+0000", 2024-09-14T07:09:43.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040082568999999998 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888274+0000", 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1869000000000002e-05 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888528+0000", 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025420500000000002 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315830674000001, 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.206 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:43.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:09:43.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.209 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315812751999999, 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:09:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:43.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315799983000002, 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315789807000002, 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315779612, 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:43.226 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.315769591999999, 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:09:43.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:09:43.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1642 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 831488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.262358738, 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039801272999999998, 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1642 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.231 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901634+0000", 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-09-14T07:09:43.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901637+0000", 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4959999999999999e-06 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901643+0000", 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4710000000000004e-06 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901648+0000", 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9320000000000004e-06 2024-09-14T07:09:43.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941196+0000", 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039548176999999997 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941234+0000", 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7966999999999997e-05 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941435+0000", 2024-09-14T07:09:43.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020081799999999999 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1643 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 854016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.262336757, 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040041757999999997, 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1643 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901656+0000", 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901658+0000", 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3190000000000002e-06 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901666+0000", 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3769999999999998e-06 2024-09-14T07:09:43.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901671+0000", 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.006e-06 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941457+0000", 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039786119000000002 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941496+0000", 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8742000000000003e-05 2024-09-14T07:09:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941697+0000", 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000201795 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1644 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 876544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.262314400000001, 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040289206000000001, 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1644 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:43.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901678+0000", 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901681+0000", 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2170000000000001e-06 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901687+0000", 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8009999999999998e-06 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901693+0000", 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941720+0000", 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040027342 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941758+0000", 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8095e-05 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941967+0000", 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000209139 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1645 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 899072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.262292066000001, 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040527821999999998, 2024-09-14T07:09:43.243 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1645 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901700+0000", 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901701+0000", 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901703+0000", 2024-09-14T07:09:43.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.176e-06 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901711+0000", 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1519999999999992e-06 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901716+0000", 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0810000000000001e-06 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.941990+0000", 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040274094000000003 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942027+0000", 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6726e-05 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942228+0000", 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020122599999999999 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.262268477999999, 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.252 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:43.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:43.260 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:43.260+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:43.362 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:43.359+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:09:43.433 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T07:09:43.449 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:43.449 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:43.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T07:09:43.463 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:43.460+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:43.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:43.564 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:43.563+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:43.617 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:43.617 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:43.617 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:43.618 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:43.618 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:43.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:43.675 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:43.675+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:43.718 INFO:teuthology.orchestra.run.smithi145.stdout:3912 trunc from 0x73bb800 to 0x9b17600 2024-09-14T07:09:43.748 INFO:teuthology.orchestra.run.smithi145.stdout:3914 trunc from 0x9b17600 to 0xc117000 2024-09-14T07:09:43.766 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:43.764+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:43.774 INFO:teuthology.orchestra.run.smithi145.stdout:3916 read 0x7cc4600 thru 0x7ccc3f0 (0x7df1 bytes) 2024-09-14T07:09:43.776 INFO:teuthology.orchestra.run.smithi145.stdout:3917 read 0x4dc5e00 thru 0x4dd23db (0xc5dc bytes) 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.108137665000001, 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.717161828000002, 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:09:43.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.717132030000002, 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:09:43.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:09:43.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.717099755, 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:43.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.717041651999999, 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.717000749999997, 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.811 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716938941999999, 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:09:43.815 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:09:43.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716904626000002, 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:09:43.819 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:09:43.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:09:43.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716885955000002, 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716861919000003, 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:09:43.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:09:43.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716841885000001, 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:09:43.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716827049999999, 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.838 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716812423999997, 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:09:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716797923000001, 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:09:43.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:09:43.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716780168, 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:43.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716765273999997, 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716751162999998, 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:43.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716729233999999, 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:43.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:09:43.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.716707923000001, 2024-09-14T07:09:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:09:43.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:43.867 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.429357021999998, 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:43.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:09:43.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:09:43.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:43.873 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:43.873 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:43.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:43.875 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:43.872+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:43.899 INFO:teuthology.orchestra.run.smithi145.stdout:3920 write 0xe4e2400 thru 0xe4f0dce (0xe9cf bytes) 2024-09-14T07:09:43.975 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:43.976+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:43.994 INFO:teuthology.orchestra.run.smithi145.stdout:3921 trunc from 0xe4f0e00 to 0x1260a00 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5467.0:40739 3.5 e109/25 3:a5f7bcff:::rbd_object_map.1161317ab2a:head v 109'484, mlcod=109'482)", 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:43.997684+0000", 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001627269, 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016692149999999999, 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997684+0000", 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997684+0000", 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997686+0000", 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.08e-06 2024-09-14T07:09:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997697+0000", 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.101e-05 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997717+0000", 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9789999999999999e-05 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997727+0000", 2024-09-14T07:09:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0214000000000001e-05 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997833+0000", 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010650999999999999 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:43.997876+0000", 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2778999999999998e-05 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:09:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:09:44.003 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:44.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:44.076 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:44.075+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:44.152 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:44.152 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:44.152 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:44.152 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:44.153 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:44.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:44.177 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:44.176+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:44.278 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:44.275+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.024394614000002, 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:09:44.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 86.979404307999999, 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:09:44.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.677471851999996, 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:44.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:44.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.356144118000003, 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:09:44.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:09:44.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:44.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:44.351 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.919015719000001, 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:44.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:09:44.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.919005061, 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:44.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:09:44.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.918990047000001, 2024-09-14T07:09:44.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:44.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.364 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.918979131999997, 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:09:44.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.918967621999997, 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:09:44.369 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:09:44.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:09:44.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.918955079, 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:09:44.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:09:44.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.912119077, 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:09:44.378 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:09:44.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:09:44.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.912108482999997, 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.912060701999998, 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:44.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.391 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.912047235000003, 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.395 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.912036262999997, 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.400 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.912025501000002, 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:09:44.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.912009447000003, 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.409 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.911992875999999, 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:09:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:09:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.911981099999998, 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:09:44.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.911970423, 2024-09-14T07:09:44.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:09:44.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:44.423 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:44.423 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:44.424 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:44.425 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:44.423+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T07:09:44.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T07:09:44.493 INFO:teuthology.orchestra.run.smithi145.stdout:3922 read 0xee2e00 thru 0xef0f25 (0xe126 bytes) 2024-09-14T07:09:44.510 INFO:teuthology.orchestra.run.smithi145.stdout:3923 read 0x322e00 thru 0x323eb8 (0x10b9 bytes) 2024-09-14T07:09:44.514 INFO:teuthology.orchestra.run.smithi145.stdout:3924 clone 28 order 22 su 65536 sc 3 2024-09-14T07:09:44.525 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:44.524+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:44.557 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:44.557 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:44.557 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:44.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:44.626 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:44.623+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:44.727 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:44.727+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:44.732 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:44.732 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:44.732 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:44.732 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:44.733 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:44.748 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:44.828 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:44.827+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:44.867 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:44.867 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:44.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:44.867 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 132.565399179, 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:44.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:44.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:44.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:44.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.671405995000001, 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.875 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:44.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:44.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.882 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.671286115999997, 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:44.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:44.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:44.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:44.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:44.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.893 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.654060995999998, 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:44.894 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:44.898 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.981466609000002, 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:09:44.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:09:44.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207424742999997, 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:09:44.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:09:44.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:09:44.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:09:44.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207392515000002, 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:09:44.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:09:44.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.913 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207370500000003, 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:09:44.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207349565000001, 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.918 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:09:44.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:09:44.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207328554999997, 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:09:44.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:09:44.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:09:44.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207294855000001, 2024-09-14T07:09:44.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207274212000002, 2024-09-14T07:09:44.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.934 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.207028995000002, 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:09:44.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:09:44.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.938 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.206997276000003, 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:09:44.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.206975516999997, 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:09:44.943 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.206943438000003, 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:09:44.947 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.205772232000001, 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:09:44.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.955 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.205749269000002, 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.956 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.205727979000002, 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:44.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:44.964 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.205707756000002, 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:44.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:44.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:44.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:44.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:44.969 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:44.970 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:44.970 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:44.968+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:45.070 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:45.068+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:45.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:45.142 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:45.142 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:45.157 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:45.171 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:45.171+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:45.269 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:45.270 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:45.272 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:45.271+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:45.286 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:45.286 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:45.304 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:45.304 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:45.318 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:45.318 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:45.318 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:45.319 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:45.319 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:45.323 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:45.334 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:45.344 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:45.364 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:45.364 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:45.380 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:45.380+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:45.442 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T07:09:45.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:09:45.461 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:45.461 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:45.471 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T07:09:45.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:45.474 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:45.472+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:45.479 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:45.565 INFO:teuthology.orchestra.run.smithi145.stdout:leaving image image_client.0-clone27 intact 2024-09-14T07:09:45.575 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:45.575+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:45.651 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:45.651 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:45.665 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:45.712 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:45.712+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:45.777 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:45.775+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:45.781 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:45.781 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:45.796 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:45.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:45.847 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-14T07:09:45.442872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1595013484,"num_objects":847,"num_object_clones":125,"num_object_copies":1694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":847,"num_whiteouts":216,"num_read":207027,"num_read_kb":701087,"num_write":11067,"num_write_kb":242493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1004,"num_bytes_recovered":128743440,"num_keys_recovered":288,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9004,"ondisk_log_size":9004,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":210276,"kb_used_data":39788,"kb_used_omap":9,"kb_used_meta":170358,"kb_avail":566020764,"statfs":{"total":579820584960,"available":579605262336,"internally_reserved":0,"allocated":40742912,"data_stored":309980122,"data_compressed":12510647,"data_compressed_allocated":27021312,"data_compressed_original":303577600,"omap_allocated":9568,"internal_metadata":174447264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,6,10],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8552813,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":42977,"num_read_kb":30099,"num_write":1092,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040554"},"pg_stats":[{"pgid":"3.1f","version":"109'145","reported_seq":5038,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.468943+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:09:44.468943+0000","last_peered":"2024-09-14T07:09:44.468943+0000","last_clean":"2024-09-14T07:09:44.468943+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:09:44.468943+0000","last_undegraded":"2024-09-14T07:09:44.468943+0000","last_fullsized":"2024-09-14T07:09:44.468943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:32.881138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":69978128,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":4446,"num_read_kb":43232,"num_write":229,"num_write_kb":10158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"110'448","reported_seq":8275,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.504941+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:44.504941+0000","last_peered":"2024-09-14T07:09:44.504941+0000","last_clean":"2024-09-14T07:09:44.504941+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:44.504941+0000","last_undegraded":"2024-09-14T07:09:44.504941+0000","last_fullsized":"2024-09-14T07:09:44.504941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:47.089420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":66759046,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":9188,"num_read_kb":39782,"num_write":732,"num_write_kb":13104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"109'476","reported_seq":7244,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448094+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:44.448094+0000","last_peered":"2024-09-14T07:09:44.448094+0000","last_clean":"2024-09-14T07:09:44.448094+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:44.448094+0000","last_undegraded":"2024-09-14T07:09:44.448094+0000","last_fullsized":"2024-09-14T07:09:44.448094+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:09.264294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":10296420,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":8933,"num_read_kb":17474,"num_write":348,"num_write_kb":2836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"110'154","reported_seq":7790,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.525020+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:44.525020+0000","last_peered":"2024-09-14T07:09:44.525020+0000","last_clean":"2024-09-14T07:09:44.525020+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:44.525020+0000","last_undegraded":"2024-09-14T07:09:44.525020+0000","last_fullsized":"2024-09-14T07:09:44.525020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:18.946104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":35413556,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":7491,"num_read_kb":13481,"num_write":170,"num_write_kb":4520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"109'377","reported_seq":7516,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.468776+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:09:44.468776+0000","last_peered":"2024-09-14T07:09:44.468776+0000","last_clean":"2024-09-14T07:09:44.468776+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:09:44.468776+0000","last_undegraded":"2024-09-14T07:09:44.468776+0000","last_fullsized":"2024-09-14T07:09:44.468776+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:35:30.978491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":102915259,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":7897,"num_read_kb":20135,"num_write":512,"num_write_kb":10477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"107'186","reported_seq":15768,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:42.183688+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:42.183688+0000","last_peered":"2024-09-14T07:09:42.183688+0000","last_clean":"2024-09-14T07:09:42.183688+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:42.183688+0000","last_undegraded":"2024-09-14T07:09:42.183688+0000","last_fullsized":"2024-09-14T07:09:42.183688+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:25:46.280038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":78739117,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":3,"num_read":16406,"num_read_kb":34412,"num_write":231,"num_write_kb":10664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"109'164","reported_seq":5211,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.436787+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:44.436787+0000","last_peered":"2024-09-14T07:09:44.436787+0000","last_clean":"2024-09-14T07:09:44.436787+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:44.436787+0000","last_undegraded":"2024-09-14T07:09:44.436787+0000","last_fullsized":"2024-09-14T07:09:44.436787+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:53:43.477657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":28751995,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":4982,"num_read_kb":12615,"num_write":184,"num_write_kb":5869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"109'168","reported_seq":2456,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.509998+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:09:44.509998+0000","last_peered":"2024-09-14T07:09:44.509998+0000","last_clean":"2024-09-14T07:09:44.509998+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:09:44.509998+0000","last_undegraded":"2024-09-14T07:09:44.509998+0000","last_fullsized":"2024-09-14T07:09:44.509998+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:46.870067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":76865694,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":2091,"num_read_kb":5649,"num_write":286,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"109'116","reported_seq":1895,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448157+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:09:44.448157+0000","last_peered":"2024-09-14T07:09:44.448157+0000","last_clean":"2024-09-14T07:09:44.448157+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:09:44.448157+0000","last_undegraded":"2024-09-14T07:09:44.448157+0000","last_fullsized":"2024-09-14T07:09:44.448157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:44:25.067675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":37352486,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1450,"num_read_kb":9211,"num_write":132,"num_write_kb":2767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"109'252","reported_seq":7092,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448152+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:44.448152+0000","last_peered":"2024-09-14T07:09:44.448152+0000","last_clean":"2024-09-14T07:09:44.448152+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:44.448152+0000","last_undegraded":"2024-09-14T07:09:44.448152+0000","last_fullsized":"2024-09-14T07:09:44.448152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:49:45.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":48148006,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":7629,"num_read_kb":32754,"num_write":224,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"110'251","reported_seq":4149,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.516375+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:09:44.516375+0000","last_peered":"2024-09-14T07:09:44.516375+0000","last_clean":"2024-09-14T07:09:44.516375+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:09:44.516375+0000","last_undegraded":"2024-09-14T07:09:44.516375+0000","last_fullsized":"2024-09-14T07:09:44.516375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:55:36.054355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":67224625,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4732,"num_read_kb":38180,"num_write":320,"num_write_kb":13909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"109'493","reported_seq":7157,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452650+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:44.452650+0000","last_peered":"2024-09-14T07:09:44.452650+0000","last_clean":"2024-09-14T07:09:44.452650+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:44.452650+0000","last_undegraded":"2024-09-14T07:09:44.452650+0000","last_fullsized":"2024-09-14T07:09:44.452650+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:36.946241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":56268915,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":8406,"num_read_kb":21190,"num_write":693,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":215,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.453249+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:44.453249+0000","last_peered":"2024-09-14T07:09:44.453249+0000","last_clean":"2024-09-14T07:09:44.453249+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:44.453249+0000","last_undegraded":"2024-09-14T07:09:44.453249+0000","last_fullsized":"2024-09-14T07:09:44.453249+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'428","reported_seq":9138,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.468888+0000","last_change":"2024-09-14T07:09:14.793462+0000","last_active":"2024-09-14T07:09:44.468888+0000","last_peered":"2024-09-14T07:09:44.468888+0000","last_clean":"2024-09-14T07:09:44.468888+0000","last_became_active":"2024-09-14T07:09:14.787124+0000","last_became_peered":"2024-09-14T07:09:14.787124+0000","last_unstale":"2024-09-14T07:09:44.468888+0000","last_undegraded":"2024-09-14T07:09:44.468888+0000","last_fullsized":"2024-09-14T07:09:44.468888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:29:10.304860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":21480210,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":6,"num_read":11194,"num_read_kb":20794,"num_write":511,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1461720,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":203,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.437383+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:44.437383+0000","last_peered":"2024-09-14T07:09:44.437383+0000","last_clean":"2024-09-14T07:09:44.437383+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:44.437383+0000","last_undegraded":"2024-09-14T07:09:44.437383+0000","last_fullsized":"2024-09-14T07:09:44.437383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"109'202","reported_seq":4255,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.450366+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:44.450366+0000","last_peered":"2024-09-14T07:09:44.450366+0000","last_clean":"2024-09-14T07:09:44.450366+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:44.450366+0000","last_undegraded":"2024-09-14T07:09:44.450366+0000","last_fullsized":"2024-09-14T07:09:44.450366+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:01.913981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":33995806,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":4590,"num_read_kb":13711,"num_write":245,"num_write_kb":2619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":215,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452873+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:09:44.452873+0000","last_peered":"2024-09-14T07:09:44.452873+0000","last_clean":"2024-09-14T07:09:44.452873+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:09:44.452873+0000","last_undegraded":"2024-09-14T07:09:44.452873+0000","last_fullsized":"2024-09-14T07:09:44.452873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"110'455","reported_seq":4531,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.515849+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:44.515849+0000","last_peered":"2024-09-14T07:09:44.515849+0000","last_clean":"2024-09-14T07:09:44.515849+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:44.515849+0000","last_undegraded":"2024-09-14T07:09:44.515849+0000","last_fullsized":"2024-09-14T07:09:44.515849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:58.945974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":4813879,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":6242,"num_read_kb":9529,"num_write":662,"num_write_kb":2070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":215,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.437327+0000","last_change":"2024-09-14T07:09:11.349379+0000","last_active":"2024-09-14T07:09:44.437327+0000","last_peered":"2024-09-14T07:09:44.437327+0000","last_clean":"2024-09-14T07:09:44.437327+0000","last_became_active":"2024-09-14T07:09:11.348941+0000","last_became_peered":"2024-09-14T07:09:11.348941+0000","last_unstale":"2024-09-14T07:09:44.437327+0000","last_undegraded":"2024-09-14T07:09:44.437327+0000","last_fullsized":"2024-09-14T07:09:44.437327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:39.931750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"110'520","reported_seq":2583,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.482906+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:44.482906+0000","last_peered":"2024-09-14T07:09:44.482906+0000","last_clean":"2024-09-14T07:09:44.482906+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:44.482906+0000","last_undegraded":"2024-09-14T07:09:44.482906+0000","last_fullsized":"2024-09-14T07:09:44.482906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:36.662195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":10432813,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":2,"num_read":4160,"num_read_kb":16465,"num_write":911,"num_write_kb":4918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"107'99","reported_seq":6151,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:41.880263+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:09:41.880263+0000","last_peered":"2024-09-14T07:09:41.880263+0000","last_clean":"2024-09-14T07:09:41.880263+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:09:41.880263+0000","last_undegraded":"2024-09-14T07:09:41.880263+0000","last_fullsized":"2024-09-14T07:09:41.880263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:17.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":64851456,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":5506,"num_read_kb":24310,"num_write":180,"num_write_kb":12044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":205,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448025+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:44.448025+0000","last_peered":"2024-09-14T07:09:44.448025+0000","last_clean":"2024-09-14T07:09:44.448025+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:44.448025+0000","last_undegraded":"2024-09-14T07:09:44.448025+0000","last_fullsized":"2024-09-14T07:09:44.448025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"109'266","reported_seq":3404,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.436742+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:44.436742+0000","last_peered":"2024-09-14T07:09:44.436742+0000","last_clean":"2024-09-14T07:09:44.436742+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:44.436742+0000","last_undegraded":"2024-09-14T07:09:44.436742+0000","last_fullsized":"2024-09-14T07:09:44.436742+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:44.429360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":27308813,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":5219,"num_read_kb":10868,"num_write":190,"num_write_kb":2734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":179,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.449899+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:44.449899+0000","last_peered":"2024-09-14T07:09:44.449899+0000","last_clean":"2024-09-14T07:09:44.449899+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:44.449899+0000","last_undegraded":"2024-09-14T07:09:44.449899+0000","last_fullsized":"2024-09-14T07:09:44.449899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"107'176","reported_seq":4043,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:41.942412+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:09:41.942412+0000","last_peered":"2024-09-14T07:09:41.942412+0000","last_clean":"2024-09-14T07:09:41.942412+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:09:41.942412+0000","last_undegraded":"2024-09-14T07:09:41.942412+0000","last_fullsized":"2024-09-14T07:09:41.942412+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:28:34.425530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":20632105,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":3686,"num_read_kb":3880,"num_write":158,"num_write_kb":1903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":203,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452536+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:44.452536+0000","last_peered":"2024-09-14T07:09:44.452536+0000","last_clean":"2024-09-14T07:09:44.452536+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:44.452536+0000","last_undegraded":"2024-09-14T07:09:44.452536+0000","last_fullsized":"2024-09-14T07:09:44.452536+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"109'183","reported_seq":4941,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.436970+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:44.436970+0000","last_peered":"2024-09-14T07:09:44.436970+0000","last_clean":"2024-09-14T07:09:44.436970+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:44.436970+0000","last_undegraded":"2024-09-14T07:09:44.436970+0000","last_fullsized":"2024-09-14T07:09:44.436970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:11.332691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":50891808,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":4595,"num_read_kb":7086,"num_write":168,"num_write_kb":2171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":224,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448112+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:44.448112+0000","last_peered":"2024-09-14T07:09:44.448112+0000","last_clean":"2024-09-14T07:09:44.448112+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:44.448112+0000","last_undegraded":"2024-09-14T07:09:44.448112+0000","last_fullsized":"2024-09-14T07:09:44.448112+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":318,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452552+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:09:44.452552+0000","last_peered":"2024-09-14T07:09:44.452552+0000","last_clean":"2024-09-14T07:09:44.452552+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:09:44.452552+0000","last_undegraded":"2024-09-14T07:09:44.452552+0000","last_fullsized":"2024-09-14T07:09:44.452552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"110'231","reported_seq":7881,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.513096+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:44.513096+0000","last_peered":"2024-09-14T07:09:44.513096+0000","last_clean":"2024-09-14T07:09:44.513096+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:44.513096+0000","last_undegraded":"2024-09-14T07:09:44.513096+0000","last_fullsized":"2024-09-14T07:09:44.513096+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:12:20.197218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":142305348,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":7395,"num_read_kb":59417,"num_write":380,"num_write_kb":25500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"107'399","reported_seq":4668,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:41.855654+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:41.855654+0000","last_peered":"2024-09-14T07:09:41.855654+0000","last_clean":"2024-09-14T07:09:41.855654+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:41.855654+0000","last_undegraded":"2024-09-14T07:09:41.855654+0000","last_fullsized":"2024-09-14T07:09:41.855654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:56:31.592039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":78286175,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":11,"num_read":5795,"num_read_kb":33317,"num_write":458,"num_write_kb":9432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"109'178","reported_seq":5795,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452701+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:44.452701+0000","last_peered":"2024-09-14T07:09:44.452701+0000","last_clean":"2024-09-14T07:09:44.452701+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:44.452701+0000","last_undegraded":"2024-09-14T07:09:44.452701+0000","last_fullsized":"2024-09-14T07:09:44.452701+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:50.155908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":33249808,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":5426,"num_read_kb":23081,"num_write":188,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"109'618","reported_seq":5531,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.437197+0000","last_change":"2024-09-14T07:09:14.798144+0000","last_active":"2024-09-14T07:09:44.437197+0000","last_peered":"2024-09-14T07:09:44.437197+0000","last_clean":"2024-09-14T07:09:44.437197+0000","last_became_active":"2024-09-14T07:09:14.796081+0000","last_became_peered":"2024-09-14T07:09:14.796081+0000","last_unstale":"2024-09-14T07:09:44.437197+0000","last_undegraded":"2024-09-14T07:09:44.437197+0000","last_fullsized":"2024-09-14T07:09:44.437197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:15:35.873761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":56984877,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":7621,"num_read_kb":28886,"num_write":904,"num_write_kb":11230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":17139225,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"110'280","reported_seq":7487,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.528803+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:09:44.528803+0000","last_peered":"2024-09-14T07:09:44.528803+0000","last_clean":"2024-09-14T07:09:44.528803+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:09:44.528803+0000","last_undegraded":"2024-09-14T07:09:44.528803+0000","last_fullsized":"2024-09-14T07:09:44.528803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:52.241630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":50191360,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":8860,"num_read_kb":20711,"num_write":200,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"109'170","reported_seq":2818,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452825+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:44.452825+0000","last_peered":"2024-09-14T07:09:44.452825+0000","last_clean":"2024-09-14T07:09:44.452825+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:44.452825+0000","last_undegraded":"2024-09-14T07:09:44.452825+0000","last_fullsized":"2024-09-14T07:09:44.452825+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:39.004553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":22191147,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2591,"num_read_kb":12743,"num_write":176,"num_write_kb":2702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"109'245","reported_seq":9233,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.526244+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:44.526244+0000","last_peered":"2024-09-14T07:09:44.526244+0000","last_clean":"2024-09-14T07:09:44.526244+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:44.526244+0000","last_undegraded":"2024-09-14T07:09:44.526244+0000","last_fullsized":"2024-09-14T07:09:44.526244+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:59.807488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":41001031,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":9876,"num_read_kb":23975,"num_write":203,"num_write_kb":4131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"110'289","reported_seq":6977,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.518214+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:44.518214+0000","last_peered":"2024-09-14T07:09:44.518214+0000","last_clean":"2024-09-14T07:09:44.518214+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:44.518214+0000","last_undegraded":"2024-09-14T07:09:44.518214+0000","last_fullsized":"2024-09-14T07:09:44.518214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:19.617131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":29534345,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":7329,"num_read_kb":12075,"num_write":450,"num_write_kb":16209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"109'167","reported_seq":5875,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.447972+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:09:44.447972+0000","last_peered":"2024-09-14T07:09:44.447972+0000","last_clean":"2024-09-14T07:09:44.447972+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:09:44.447972+0000","last_undegraded":"2024-09-14T07:09:44.447972+0000","last_fullsized":"2024-09-14T07:09:44.447972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:14.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":47210258,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":5192,"num_read_kb":16760,"num_write":275,"num_write_kb":15559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"109'306","reported_seq":7265,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452585+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:44.452585+0000","last_peered":"2024-09-14T07:09:44.452585+0000","last_clean":"2024-09-14T07:09:44.452585+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:44.452585+0000","last_undegraded":"2024-09-14T07:09:44.452585+0000","last_fullsized":"2024-09-14T07:09:44.452585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:46:36.371287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":59253391,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":7719,"num_read_kb":17127,"num_write":243,"num_write_kb":7138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"110'287","reported_seq":5733,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.520437+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:09:44.520437+0000","last_peered":"2024-09-14T07:09:44.520437+0000","last_clean":"2024-09-14T07:09:44.520437+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:09:44.520437+0000","last_undegraded":"2024-09-14T07:09:44.520437+0000","last_fullsized":"2024-09-14T07:09:44.520437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:48:27.064967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":55804416,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":6011,"num_read_kb":20352,"num_write":197,"num_write_kb":6542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"110'200","reported_seq":2787,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.504772+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:44.504772+0000","last_peered":"2024-09-14T07:09:44.504772+0000","last_clean":"2024-09-14T07:09:44.504772+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:44.504772+0000","last_undegraded":"2024-09-14T07:09:44.504772+0000","last_fullsized":"2024-09-14T07:09:44.504772+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:41:01.203831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":65290804,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":4303,"num_read_kb":37830,"num_write":204,"num_write_kb":4759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1594423097,"num_objects":843,"num_object_clones":125,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":216,"num_read":206961,"num_read_kb":701032,"num_write":10964,"num_write_kb":240863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":127562704,"num_keys_recovered":288,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32960512,"data_stored":305745010,"data_compressed":12498620,"data_compressed_allocated":26955776,"data_compressed_original":302463488,"omap_allocated":0,"internal_metadata":0},"log_size":8929,"ondisk_log_size":8929,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39168,"kb_used_data":7984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332672,"statfs":{"total":96636764160,"available":96596656128,"internally_reserved":0,"allocated":8175616,"data_stored":56445634,"data_compressed":2700794,"data_compressed_allocated":5537792,"data_compressed_original":55203328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607583,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":4744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":4857856,"data_stored":21340006,"data_compressed":1692564,"data_compressed_allocated":2813952,"data_compressed_original":20412416,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607582,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37848,"kb_used_data":6660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333992,"statfs":{"total":96636764160,"available":96598007808,"internally_reserved":0,"allocated":6819840,"data_stored":54909815,"data_compressed":2099925,"data_compressed_allocated":4571136,"data_compressed_original":53846528,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,10],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607584,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":6132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":6279168,"data_stored":46992933,"data_compressed":1906039,"data_compressed_allocated":4055040,"data_compressed_original":45998592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607584,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":6300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":6451200,"data_stored":59657496,"data_compressed":1654693,"data_compressed_allocated":4300800,"data_compressed_original":58705920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607584,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34996,"kb_used_data":7968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336844,"statfs":{"total":96636764160,"available":96600928256,"internally_reserved":0,"allocated":8159232,"data_stored":70634238,"data_compressed":2456632,"data_compressed_allocated":5742592,"data_compressed_original":69410816,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6819840,"data_stored":69519162,"data_compressed":2449740,"data_compressed_allocated":5705728,"data_compressed_original":68820992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5148672,"data_stored":59132769,"data_compressed":1654693,"data_compressed_allocated":4300800,"data_compressed_original":58705920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4980736,"data_stored":46468225,"data_compressed":1906039,"data_compressed_allocated":4055040,"data_compressed_original":45998592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5464064,"data_stored":53794739,"data_compressed":2093867,"data_compressed_allocated":4534272,"data_compressed_original":53273088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6979584,"data_stored":56010076,"data_compressed":2701717,"data_compressed_allocated":5545984,"data_compressed_original":55252480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3567616,"data_stored":20820039,"data_compressed":1692564,"data_compressed_allocated":2813952,"data_compressed_original":20412416,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:45.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:45.895 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:45.895+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:45.969 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:45.969 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:45.980 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:45.980+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:45.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:46.081 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:46.079+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:46.114 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T07:09:46.119 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:46.119 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:46.134 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:46.182 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:46.179+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:46.211 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:46.229 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-14T07:09:45.442872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1595013484,"num_objects":847,"num_object_clones":125,"num_object_copies":1694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":847,"num_whiteouts":216,"num_read":207027,"num_read_kb":701087,"num_write":11067,"num_write_kb":242493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1004,"num_bytes_recovered":128743440,"num_keys_recovered":288,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9004,"ondisk_log_size":9004,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":210276,"kb_used_data":39788,"kb_used_omap":9,"kb_used_meta":170358,"kb_avail":566020764,"statfs":{"total":579820584960,"available":579605262336,"internally_reserved":0,"allocated":40742912,"data_stored":309980122,"data_compressed":12510647,"data_compressed_allocated":27021312,"data_compressed_original":303577600,"omap_allocated":9568,"internal_metadata":174447264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,6,10],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8552813,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":42977,"num_read_kb":30099,"num_write":1092,"num_write_kb":10562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040554"},"pg_stats":[{"pgid":"3.1f","version":"109'145","reported_seq":5038,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.468943+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:09:44.468943+0000","last_peered":"2024-09-14T07:09:44.468943+0000","last_clean":"2024-09-14T07:09:44.468943+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:09:44.468943+0000","last_undegraded":"2024-09-14T07:09:44.468943+0000","last_fullsized":"2024-09-14T07:09:44.468943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:32.881138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":69978128,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":4446,"num_read_kb":43232,"num_write":229,"num_write_kb":10158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"110'448","reported_seq":8275,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.504941+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:44.504941+0000","last_peered":"2024-09-14T07:09:44.504941+0000","last_clean":"2024-09-14T07:09:44.504941+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:44.504941+0000","last_undegraded":"2024-09-14T07:09:44.504941+0000","last_fullsized":"2024-09-14T07:09:44.504941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:47.089420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":66759046,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":9188,"num_read_kb":39782,"num_write":732,"num_write_kb":13104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"109'476","reported_seq":7244,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448094+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:44.448094+0000","last_peered":"2024-09-14T07:09:44.448094+0000","last_clean":"2024-09-14T07:09:44.448094+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:44.448094+0000","last_undegraded":"2024-09-14T07:09:44.448094+0000","last_fullsized":"2024-09-14T07:09:44.448094+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:09.264294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":10296420,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":8933,"num_read_kb":17474,"num_write":348,"num_write_kb":2836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"110'154","reported_seq":7790,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.525020+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:44.525020+0000","last_peered":"2024-09-14T07:09:44.525020+0000","last_clean":"2024-09-14T07:09:44.525020+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:44.525020+0000","last_undegraded":"2024-09-14T07:09:44.525020+0000","last_fullsized":"2024-09-14T07:09:44.525020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:18.946104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":35413556,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":2,"num_read":7491,"num_read_kb":13481,"num_write":170,"num_write_kb":4520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"109'377","reported_seq":7516,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.468776+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:09:44.468776+0000","last_peered":"2024-09-14T07:09:44.468776+0000","last_clean":"2024-09-14T07:09:44.468776+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:09:44.468776+0000","last_undegraded":"2024-09-14T07:09:44.468776+0000","last_fullsized":"2024-09-14T07:09:44.468776+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:35:30.978491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":102915259,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":7897,"num_read_kb":20135,"num_write":512,"num_write_kb":10477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"107'186","reported_seq":15768,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:42.183688+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:42.183688+0000","last_peered":"2024-09-14T07:09:42.183688+0000","last_clean":"2024-09-14T07:09:42.183688+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:42.183688+0000","last_undegraded":"2024-09-14T07:09:42.183688+0000","last_fullsized":"2024-09-14T07:09:42.183688+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:25:46.280038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":78739117,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":3,"num_read":16406,"num_read_kb":34412,"num_write":231,"num_write_kb":10664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"109'164","reported_seq":5211,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.436787+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:44.436787+0000","last_peered":"2024-09-14T07:09:44.436787+0000","last_clean":"2024-09-14T07:09:44.436787+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:44.436787+0000","last_undegraded":"2024-09-14T07:09:44.436787+0000","last_fullsized":"2024-09-14T07:09:44.436787+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:53:43.477657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":28751995,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":4982,"num_read_kb":12615,"num_write":184,"num_write_kb":5869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"109'168","reported_seq":2456,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.509998+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:09:44.509998+0000","last_peered":"2024-09-14T07:09:44.509998+0000","last_clean":"2024-09-14T07:09:44.509998+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:09:44.509998+0000","last_undegraded":"2024-09-14T07:09:44.509998+0000","last_fullsized":"2024-09-14T07:09:44.509998+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:46.870067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":76865694,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":2091,"num_read_kb":5649,"num_write":286,"num_write_kb":13842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"109'116","reported_seq":1895,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448157+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:09:44.448157+0000","last_peered":"2024-09-14T07:09:44.448157+0000","last_clean":"2024-09-14T07:09:44.448157+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:09:44.448157+0000","last_undegraded":"2024-09-14T07:09:44.448157+0000","last_fullsized":"2024-09-14T07:09:44.448157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:44:25.067675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":37352486,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1450,"num_read_kb":9211,"num_write":132,"num_write_kb":2767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"109'252","reported_seq":7092,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448152+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:44.448152+0000","last_peered":"2024-09-14T07:09:44.448152+0000","last_clean":"2024-09-14T07:09:44.448152+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:44.448152+0000","last_undegraded":"2024-09-14T07:09:44.448152+0000","last_fullsized":"2024-09-14T07:09:44.448152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:49:45.183276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":48148006,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":7629,"num_read_kb":32754,"num_write":224,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"110'251","reported_seq":4149,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.516375+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:09:44.516375+0000","last_peered":"2024-09-14T07:09:44.516375+0000","last_clean":"2024-09-14T07:09:44.516375+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:09:44.516375+0000","last_undegraded":"2024-09-14T07:09:44.516375+0000","last_fullsized":"2024-09-14T07:09:44.516375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:55:36.054355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":67224625,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4732,"num_read_kb":38180,"num_write":320,"num_write_kb":13909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"109'493","reported_seq":7157,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452650+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:44.452650+0000","last_peered":"2024-09-14T07:09:44.452650+0000","last_clean":"2024-09-14T07:09:44.452650+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:44.452650+0000","last_undegraded":"2024-09-14T07:09:44.452650+0000","last_fullsized":"2024-09-14T07:09:44.452650+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:36.946241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":56268915,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":8406,"num_read_kb":21190,"num_write":693,"num_write_kb":4818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":215,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.453249+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:44.453249+0000","last_peered":"2024-09-14T07:09:44.453249+0000","last_clean":"2024-09-14T07:09:44.453249+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:44.453249+0000","last_undegraded":"2024-09-14T07:09:44.453249+0000","last_fullsized":"2024-09-14T07:09:44.453249+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'428","reported_seq":9138,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.468888+0000","last_change":"2024-09-14T07:09:14.793462+0000","last_active":"2024-09-14T07:09:44.468888+0000","last_peered":"2024-09-14T07:09:44.468888+0000","last_clean":"2024-09-14T07:09:44.468888+0000","last_became_active":"2024-09-14T07:09:14.787124+0000","last_became_peered":"2024-09-14T07:09:14.787124+0000","last_unstale":"2024-09-14T07:09:44.468888+0000","last_undegraded":"2024-09-14T07:09:44.468888+0000","last_fullsized":"2024-09-14T07:09:44.468888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:29:10.304860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":21480210,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":6,"num_read":11194,"num_read_kb":20794,"num_write":511,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1461720,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":203,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.437383+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:44.437383+0000","last_peered":"2024-09-14T07:09:44.437383+0000","last_clean":"2024-09-14T07:09:44.437383+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:44.437383+0000","last_undegraded":"2024-09-14T07:09:44.437383+0000","last_fullsized":"2024-09-14T07:09:44.437383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"109'202","reported_seq":4255,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.450366+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:44.450366+0000","last_peered":"2024-09-14T07:09:44.450366+0000","last_clean":"2024-09-14T07:09:44.450366+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:44.450366+0000","last_undegraded":"2024-09-14T07:09:44.450366+0000","last_fullsized":"2024-09-14T07:09:44.450366+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:01.913981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":33995806,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":4590,"num_read_kb":13711,"num_write":245,"num_write_kb":2619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":215,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452873+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:09:44.452873+0000","last_peered":"2024-09-14T07:09:44.452873+0000","last_clean":"2024-09-14T07:09:44.452873+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:09:44.452873+0000","last_undegraded":"2024-09-14T07:09:44.452873+0000","last_fullsized":"2024-09-14T07:09:44.452873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"110'455","reported_seq":4531,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.515849+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:44.515849+0000","last_peered":"2024-09-14T07:09:44.515849+0000","last_clean":"2024-09-14T07:09:44.515849+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:44.515849+0000","last_undegraded":"2024-09-14T07:09:44.515849+0000","last_fullsized":"2024-09-14T07:09:44.515849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:58.945974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":4813879,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":6242,"num_read_kb":9529,"num_write":662,"num_write_kb":2070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":215,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.437327+0000","last_change":"2024-09-14T07:09:11.349379+0000","last_active":"2024-09-14T07:09:44.437327+0000","last_peered":"2024-09-14T07:09:44.437327+0000","last_clean":"2024-09-14T07:09:44.437327+0000","last_became_active":"2024-09-14T07:09:11.348941+0000","last_became_peered":"2024-09-14T07:09:11.348941+0000","last_unstale":"2024-09-14T07:09:44.437327+0000","last_undegraded":"2024-09-14T07:09:44.437327+0000","last_fullsized":"2024-09-14T07:09:44.437327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:39.931750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"110'520","reported_seq":2583,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.482906+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:44.482906+0000","last_peered":"2024-09-14T07:09:44.482906+0000","last_clean":"2024-09-14T07:09:44.482906+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:44.482906+0000","last_undegraded":"2024-09-14T07:09:44.482906+0000","last_fullsized":"2024-09-14T07:09:44.482906+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:36.662195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":10432813,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":2,"num_read":4160,"num_read_kb":16465,"num_write":911,"num_write_kb":4918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"107'99","reported_seq":6151,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:41.880263+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:09:41.880263+0000","last_peered":"2024-09-14T07:09:41.880263+0000","last_clean":"2024-09-14T07:09:41.880263+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:09:41.880263+0000","last_undegraded":"2024-09-14T07:09:41.880263+0000","last_fullsized":"2024-09-14T07:09:41.880263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:17.038196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":64851456,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":5506,"num_read_kb":24310,"num_write":180,"num_write_kb":12044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":205,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448025+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:44.448025+0000","last_peered":"2024-09-14T07:09:44.448025+0000","last_clean":"2024-09-14T07:09:44.448025+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:44.448025+0000","last_undegraded":"2024-09-14T07:09:44.448025+0000","last_fullsized":"2024-09-14T07:09:44.448025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"109'266","reported_seq":3404,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.436742+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:44.436742+0000","last_peered":"2024-09-14T07:09:44.436742+0000","last_clean":"2024-09-14T07:09:44.436742+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:44.436742+0000","last_undegraded":"2024-09-14T07:09:44.436742+0000","last_fullsized":"2024-09-14T07:09:44.436742+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:44.429360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":27308813,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":5219,"num_read_kb":10868,"num_write":190,"num_write_kb":2734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":179,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.449899+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:44.449899+0000","last_peered":"2024-09-14T07:09:44.449899+0000","last_clean":"2024-09-14T07:09:44.449899+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:44.449899+0000","last_undegraded":"2024-09-14T07:09:44.449899+0000","last_fullsized":"2024-09-14T07:09:44.449899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"107'176","reported_seq":4043,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:41.942412+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:09:41.942412+0000","last_peered":"2024-09-14T07:09:41.942412+0000","last_clean":"2024-09-14T07:09:41.942412+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:09:41.942412+0000","last_undegraded":"2024-09-14T07:09:41.942412+0000","last_fullsized":"2024-09-14T07:09:41.942412+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:28:34.425530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":20632105,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":9,"num_read":3686,"num_read_kb":3880,"num_write":158,"num_write_kb":1903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":203,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452536+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:44.452536+0000","last_peered":"2024-09-14T07:09:44.452536+0000","last_clean":"2024-09-14T07:09:44.452536+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:44.452536+0000","last_undegraded":"2024-09-14T07:09:44.452536+0000","last_fullsized":"2024-09-14T07:09:44.452536+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"109'183","reported_seq":4941,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.436970+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:44.436970+0000","last_peered":"2024-09-14T07:09:44.436970+0000","last_clean":"2024-09-14T07:09:44.436970+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:44.436970+0000","last_undegraded":"2024-09-14T07:09:44.436970+0000","last_fullsized":"2024-09-14T07:09:44.436970+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:11.332691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":50891808,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":4595,"num_read_kb":7086,"num_write":168,"num_write_kb":2171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":224,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.448112+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:44.448112+0000","last_peered":"2024-09-14T07:09:44.448112+0000","last_clean":"2024-09-14T07:09:44.448112+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:44.448112+0000","last_undegraded":"2024-09-14T07:09:44.448112+0000","last_fullsized":"2024-09-14T07:09:44.448112+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":318,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452552+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:09:44.452552+0000","last_peered":"2024-09-14T07:09:44.452552+0000","last_clean":"2024-09-14T07:09:44.452552+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:09:44.452552+0000","last_undegraded":"2024-09-14T07:09:44.452552+0000","last_fullsized":"2024-09-14T07:09:44.452552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"110'231","reported_seq":7881,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.513096+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:44.513096+0000","last_peered":"2024-09-14T07:09:44.513096+0000","last_clean":"2024-09-14T07:09:44.513096+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:44.513096+0000","last_undegraded":"2024-09-14T07:09:44.513096+0000","last_fullsized":"2024-09-14T07:09:44.513096+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:12:20.197218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":142305348,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":4,"num_read":7395,"num_read_kb":59417,"num_write":380,"num_write_kb":25500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"107'399","reported_seq":4668,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-14T07:09:41.855654+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:41.855654+0000","last_peered":"2024-09-14T07:09:41.855654+0000","last_clean":"2024-09-14T07:09:41.855654+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:41.855654+0000","last_undegraded":"2024-09-14T07:09:41.855654+0000","last_fullsized":"2024-09-14T07:09:41.855654+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:56:31.592039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":78286175,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":11,"num_read":5795,"num_read_kb":33317,"num_write":458,"num_write_kb":9432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"109'178","reported_seq":5795,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452701+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:44.452701+0000","last_peered":"2024-09-14T07:09:44.452701+0000","last_clean":"2024-09-14T07:09:44.452701+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:44.452701+0000","last_undegraded":"2024-09-14T07:09:44.452701+0000","last_fullsized":"2024-09-14T07:09:44.452701+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:50.155908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":33249808,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":5426,"num_read_kb":23081,"num_write":188,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"109'618","reported_seq":5531,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.437197+0000","last_change":"2024-09-14T07:09:14.798144+0000","last_active":"2024-09-14T07:09:44.437197+0000","last_peered":"2024-09-14T07:09:44.437197+0000","last_clean":"2024-09-14T07:09:44.437197+0000","last_became_active":"2024-09-14T07:09:14.796081+0000","last_became_peered":"2024-09-14T07:09:14.796081+0000","last_unstale":"2024-09-14T07:09:44.437197+0000","last_undegraded":"2024-09-14T07:09:44.437197+0000","last_fullsized":"2024-09-14T07:09:44.437197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:15:35.873761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":56984877,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":7621,"num_read_kb":28886,"num_write":904,"num_write_kb":11230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":17139225,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"110'280","reported_seq":7487,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.528803+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:09:44.528803+0000","last_peered":"2024-09-14T07:09:44.528803+0000","last_clean":"2024-09-14T07:09:44.528803+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:09:44.528803+0000","last_undegraded":"2024-09-14T07:09:44.528803+0000","last_fullsized":"2024-09-14T07:09:44.528803+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":280,"log_dups_size":0,"ondisk_log_size":280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:52.241630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":50191360,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":8860,"num_read_kb":20711,"num_write":200,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"109'170","reported_seq":2818,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452825+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:44.452825+0000","last_peered":"2024-09-14T07:09:44.452825+0000","last_clean":"2024-09-14T07:09:44.452825+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:44.452825+0000","last_undegraded":"2024-09-14T07:09:44.452825+0000","last_fullsized":"2024-09-14T07:09:44.452825+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:39.004553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":22191147,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2591,"num_read_kb":12743,"num_write":176,"num_write_kb":2702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"109'245","reported_seq":9233,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.526244+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:44.526244+0000","last_peered":"2024-09-14T07:09:44.526244+0000","last_clean":"2024-09-14T07:09:44.526244+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:44.526244+0000","last_undegraded":"2024-09-14T07:09:44.526244+0000","last_fullsized":"2024-09-14T07:09:44.526244+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:59.807488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":41001031,"num_objects":19,"num_object_clones":2,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":9876,"num_read_kb":23975,"num_write":203,"num_write_kb":4131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"110'289","reported_seq":6977,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.518214+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:44.518214+0000","last_peered":"2024-09-14T07:09:44.518214+0000","last_clean":"2024-09-14T07:09:44.518214+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:44.518214+0000","last_undegraded":"2024-09-14T07:09:44.518214+0000","last_fullsized":"2024-09-14T07:09:44.518214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:19.617131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":29534345,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":7329,"num_read_kb":12075,"num_write":450,"num_write_kb":16209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"109'167","reported_seq":5875,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.447972+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:09:44.447972+0000","last_peered":"2024-09-14T07:09:44.447972+0000","last_clean":"2024-09-14T07:09:44.447972+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:09:44.447972+0000","last_undegraded":"2024-09-14T07:09:44.447972+0000","last_fullsized":"2024-09-14T07:09:44.447972+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:58:14.018475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":47210258,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":5192,"num_read_kb":16760,"num_write":275,"num_write_kb":15559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"109'306","reported_seq":7265,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.452585+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:44.452585+0000","last_peered":"2024-09-14T07:09:44.452585+0000","last_clean":"2024-09-14T07:09:44.452585+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:44.452585+0000","last_undegraded":"2024-09-14T07:09:44.452585+0000","last_fullsized":"2024-09-14T07:09:44.452585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:46:36.371287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":59253391,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":7719,"num_read_kb":17127,"num_write":243,"num_write_kb":7138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"110'287","reported_seq":5733,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.520437+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:09:44.520437+0000","last_peered":"2024-09-14T07:09:44.520437+0000","last_clean":"2024-09-14T07:09:44.520437+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:09:44.520437+0000","last_undegraded":"2024-09-14T07:09:44.520437+0000","last_fullsized":"2024-09-14T07:09:44.520437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:48:27.064967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":55804416,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":6011,"num_read_kb":20352,"num_write":197,"num_write_kb":6542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"110'200","reported_seq":2787,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:09:44.504772+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:44.504772+0000","last_peered":"2024-09-14T07:09:44.504772+0000","last_clean":"2024-09-14T07:09:44.504772+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:44.504772+0000","last_undegraded":"2024-09-14T07:09:44.504772+0000","last_fullsized":"2024-09-14T07:09:44.504772+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:41:01.203831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":65290804,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":4303,"num_read_kb":37830,"num_write":204,"num_write_kb":4759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1594423097,"num_objects":843,"num_object_clones":125,"num_object_copies":1686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":216,"num_read":206961,"num_read_kb":701032,"num_write":10964,"num_write_kb":240863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":127562704,"num_keys_recovered":288,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32960512,"data_stored":305745010,"data_compressed":12498620,"data_compressed_allocated":26955776,"data_compressed_original":302463488,"omap_allocated":0,"internal_metadata":0},"log_size":8929,"ondisk_log_size":8929,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39168,"kb_used_data":7984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332672,"statfs":{"total":96636764160,"available":96596656128,"internally_reserved":0,"allocated":8175616,"data_stored":56445634,"data_compressed":2700794,"data_compressed_allocated":5537792,"data_compressed_original":55203328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607583,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31768,"kb_used_data":4744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":4857856,"data_stored":21340006,"data_compressed":1692564,"data_compressed_allocated":2813952,"data_compressed_original":20412416,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607582,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37848,"kb_used_data":6660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333992,"statfs":{"total":96636764160,"available":96598007808,"internally_reserved":0,"allocated":6819840,"data_stored":54909815,"data_compressed":2099925,"data_compressed_allocated":4571136,"data_compressed_original":53846528,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,10],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607584,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33160,"kb_used_data":6132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338680,"statfs":{"total":96636764160,"available":96602808320,"internally_reserved":0,"allocated":6279168,"data_stored":46992933,"data_compressed":1906039,"data_compressed_allocated":4055040,"data_compressed_original":45998592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607584,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33336,"kb_used_data":6300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338504,"statfs":{"total":96636764160,"available":96602628096,"internally_reserved":0,"allocated":6451200,"data_stored":59657496,"data_compressed":1654693,"data_compressed_allocated":4300800,"data_compressed_original":58705920,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607584,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34996,"kb_used_data":7968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336844,"statfs":{"total":96636764160,"available":96600928256,"internally_reserved":0,"allocated":8159232,"data_stored":70634238,"data_compressed":2456632,"data_compressed_allocated":5742592,"data_compressed_original":69410816,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6819840,"data_stored":69519162,"data_compressed":2449740,"data_compressed_allocated":5705728,"data_compressed_original":68820992,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5148672,"data_stored":59132769,"data_compressed":1654693,"data_compressed_allocated":4300800,"data_compressed_original":58705920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4980736,"data_stored":46468225,"data_compressed":1906039,"data_compressed_allocated":4055040,"data_compressed_original":45998592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5464064,"data_stored":53794739,"data_compressed":2093867,"data_compressed_allocated":4534272,"data_compressed_original":53273088,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6979584,"data_stored":56010076,"data_compressed":2701717,"data_compressed_allocated":5545984,"data_compressed_original":55252480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3567616,"data_stored":20820039,"data_compressed":1692564,"data_compressed_allocated":2813952,"data_compressed_original":20412416,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:46.230 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T07:09:46.231 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:46.231 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-14T07:09:46.231 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:09:46.231 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:09:46.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-14T07:09:46.282 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:46.279+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:46.296 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:46.296 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:46.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:46.371 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:46.371 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "" 2024-09-14T07:09:46.371 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:46.383 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:46.379+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:46.438 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:46.438 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:46.449 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:09:46.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:46.523 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:46.523+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:46.574 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:46.574 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:46.584 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:46.583+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:46.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:46.685 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:46.683+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:46.760 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:46.760 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:46.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:46.815 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:46.815+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:46.887 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:46.883+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:46.938 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:46.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:46.987 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:46.987+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:47.088 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:47.087+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:47.125 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:47.125 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:47.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:47.189 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:47.187+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:47.264 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:47.264 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:47.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:47.323 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:47.323+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:47.334 INFO:teuthology.orchestra.run.smithi145.stdout:3928 trunc from 0x1260a00 to 0x6513a00 2024-09-14T07:09:47.365 INFO:teuthology.orchestra.run.smithi145.stdout:3929 write 0x3416a00 thru 0x3422c14 (0xc215 bytes) 2024-09-14T07:09:47.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:47.409 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:47.410 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:47.410 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:47.407+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:47.425 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:47.457 INFO:teuthology.orchestra.run.smithi145.stdout:3930 write 0x860c000 thru 0x860d7ef (0x17f0 bytes) 2024-09-14T07:09:47.460 INFO:teuthology.orchestra.run.smithi145.stdout:3932 punch from 0x3c6f400 to 0x3c73600, (0x4200 bytes) 2024-09-14T07:09:47.473 INFO:teuthology.orchestra.run.smithi145.stdout:3933 trunc from 0x860d800 to 0x34e0c00 2024-09-14T07:09:47.491 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:47.491+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:47.544 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:47.544 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:47.561 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:47.574 INFO:teuthology.orchestra.run.smithi145.stdout:3934 write 0x1e18c00 thru 0x1e1b26d (0x266e bytes) 2024-09-14T07:09:47.577 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:47.577 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:47.578 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:47.592 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:47.591+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:47.594 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:47.597 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:47.598 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:47.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:47.633 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:47.670 INFO:teuthology.orchestra.run.smithi145.stdout:3940 write 0xdacb000 thru 0xdad5255 (0xa256 bytes) 2024-09-14T07:09:47.694 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:47.691+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:47.696 INFO:teuthology.orchestra.run.smithi145.stdout:3944 trunc from 0xdad5400 to 0x3279a00 2024-09-14T07:09:47.772 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:47.772 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:47.772 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:47.772 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:47.772 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:47.787 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:47.832 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:47.831+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:47.895 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:47.895+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.62050907899999, 2024-09-14T07:09:47.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:47.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:47.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:47.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.726515894999999, 2024-09-14T07:09:47.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:47.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.726396016000002, 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:47.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:47.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.709170896000003, 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:47.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:47.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.036576509, 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:47.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:47.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:09:47.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:09:47.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262534643000002, 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:47.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:09:47.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262502415, 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:09:47.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262480400000001, 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:09:47.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262459464999999, 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.975 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:09:47.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262438455000002, 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:09:47.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:09:47.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:47.983 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262404754999999, 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:47.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:09:47.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262384111999999, 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:09:47.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:09:47.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262138895, 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:47.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:09:47.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:09:47.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:47.996 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262107176000001, 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:47.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:09:47.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:47.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262085417000002, 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:09:48.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.262053338000001, 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.005 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.260882131999999, 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.010 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:09:48.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:09:48.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:48.014 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.260859169, 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.018 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.260837879, 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.019 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:09:48.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.260817656, 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.023 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.027 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:48.029 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:48.030 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:48.027+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:48.050 INFO:teuthology.orchestra.run.smithi145.stdout:3947 read 0x1f41200 thru 0x1f457ab (0x45ac bytes) 2024-09-14T07:09:48.050 INFO:teuthology.orchestra.run.smithi145.stdout:3948 punch from 0x55e200 to 0x561000, (0x2e00 bytes) 2024-09-14T07:09:48.129 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:48.127+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:48.230 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:48.227+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:48.249 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:48.250 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:48.250 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:48.250 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:48.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:48.331 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:48.331+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:48.432 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:48.431+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:48.464 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:48.464 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:48.464 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:48.464 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:48.464 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:48.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:09:48.484 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:48.529 INFO:teuthology.orchestra.run.smithi145.stdout:3951 write 0x641000 thru 0x64712e (0x612f bytes) 2024-09-14T07:09:48.572 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:48.571+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:48.633 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:48.631+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:48.644 INFO:teuthology.orchestra.run.smithi145.stdout:3954 punch from 0x1ade00 to 0x1bbe00, (0xe000 bytes) 2024-09-14T07:09:48.674 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 91.377182915999995, 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.517707329000004, 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.517605044000007, 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.690 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 77.818553202999993, 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:48.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:48.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.695 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333632010999999, 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:09:48.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:09:48.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333596790999998, 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.700 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.704 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333541642, 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:09:48.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333507572999999, 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:09:48.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333464227, 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.713 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:09:48.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:09:48.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333430704000001, 2024-09-14T07:09:48.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:09:48.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:09:48.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:48.721 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333383454, 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:48.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.725 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333269205999997, 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:09:48.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333159510999998, 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.333093368, 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.734 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:48.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:09:48.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.332976864000003, 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:09:48.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.332911207999999, 2024-09-14T07:09:48.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:09:48.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:09:48.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.332851888, 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:09:48.746 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.332711019999998, 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:09:48.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.332295913000003, 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:48.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:09:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.332228174000001, 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.759 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.762 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.763 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.763 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:48.764 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:48.764 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:48.763+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:48.865 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:48.863+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:48.881 INFO:teuthology.orchestra.run.smithi145.stdout:3955 punch from 0x4e8000 to 0x4ed400, (0x5400 bytes) 2024-09-14T07:09:48.962 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12108 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.945345+0000", 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.014495506, 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014517008, 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12108 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945345+0000", 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945345+0000", 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945345+0000", 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5700000000000002e-07 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945348+0000", 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945354+0000", 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3829999999999997e-06 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945359+0000", 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9060000000000001e-06 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12140 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.946093+0000", 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.013747538, 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013827186999999999, 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12140 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946093+0000", 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946093+0000", 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946093+0000", 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1500000000000004e-07 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946096+0000", 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3240000000000001e-06 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946102+0000", 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4420000000000003e-06 2024-09-14T07:09:48.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946107+0000", 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.87e-06 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12172 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.952532+0000", 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0073085329999999999, 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0074372989999999996, 2024-09-14T07:09:48.969 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12172 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952532+0000", 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952532+0000", 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952532+0000", 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952535+0000", 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8559999999999998e-06 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952544+0000", 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1849999999999993e-06 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952550+0000", 2024-09-14T07:09:48.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0270000000000001e-06 2024-09-14T07:09:48.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.973 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12109 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.945366+0000", 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.014474305999999999, 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014655321000000001, 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.977 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12109 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945366+0000", 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945366+0000", 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945366+0000", 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4000000000000002e-07 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945368+0000", 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.08e-06 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945375+0000", 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4489999999999996e-06 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.945380+0000", 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12141 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.946113+0000", 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.013727069, 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013957054, 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12141 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946113+0000", 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946113+0000", 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946114+0000", 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946116+0000", 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.934e-06 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946122+0000", 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1009999999999997e-06 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946127+0000", 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9300000000000002e-06 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.983 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12173 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.952559+0000", 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0072816740000000001, 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0075609650000000002, 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12173 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952559+0000", 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952559+0000", 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952559+0000", 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8100000000000003e-07 2024-09-14T07:09:48.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952561+0000", 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1430000000000001e-06 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952568+0000", 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952574+0000", 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0599999999999998e-06 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12142 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.946133+0000", 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.013706846999999999, 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014035598999999999, 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12142 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946133+0000", 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946133+0000", 2024-09-14T07:09:48.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946134+0000", 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7100000000000002e-07 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946136+0000", 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946142+0000", 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1630000000000001e-06 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946147+0000", 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12174 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.990 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.952583+0000", 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0072576899999999998, 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0076350699999999999, 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12174 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952583+0000", 2024-09-14T07:09:48.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952583+0000", 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952583+0000", 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.68e-07 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952585+0000", 2024-09-14T07:09:48.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952592+0000", 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3459999999999999e-06 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952597+0000", 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8620000000000002e-06 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12143 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.946154+0000", 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.013685767999999999, 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014123501, 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12143 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946154+0000", 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946154+0000", 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946155+0000", 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946157+0000", 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9999999999999999e-06 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946163+0000", 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3130000000000004e-06 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946168+0000", 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7230000000000004e-06 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:48.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12175 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.952604+0000", 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.007236508, 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077247710000000001, 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12175 2024-09-14T07:09:48.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952604+0000", 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952604+0000", 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952604+0000", 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:09:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952606+0000", 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8840000000000001e-06 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952614+0000", 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5220000000000001e-06 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952618+0000", 2024-09-14T07:09:48.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12144 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.946174+0000", 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.013665988, 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014205789, 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12144 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946174+0000", 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946174+0000", 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946175+0000", 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5299999999999999e-07 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946177+0000", 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9759999999999998e-06 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946184+0000", 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8419999999999999e-06 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946188+0000", 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8810000000000004e-06 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12176 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.003 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.952625+0000", 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0072149780000000004, 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0078064049999999998, 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12176 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952625+0000", 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952625+0000", 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952626+0000", 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.68e-07 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952628+0000", 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9769999999999999e-06 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952634+0000", 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.032e-06 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952639+0000", 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8359999999999999e-06 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.006 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12145 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.946195+0000", 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.013645717999999999, 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.014288275, 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.007 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12145 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946195+0000", 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946195+0000", 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946195+0000", 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946197+0000", 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946203+0000", 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6720000000000001e-06 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.946207+0000", 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7559999999999999e-06 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12177 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.952655+0000", 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0071849549999999998, 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.007878309, 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.010 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12177 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952655+0000", 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952655+0000", 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952656+0000", 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9400000000000005e-07 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952658+0000", 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.396e-06 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952665+0000", 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952672+0000", 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8390000000000001e-06 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:12178 3.c 3.677d474c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.013 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.952680+0000", 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.007160235, 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0079033539999999996, 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 12178 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952680+0000", 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952680+0000", 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952681+0000", 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1600000000000001e-07 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952683+0000", 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4099999999999998e-06 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952690+0000", 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6610000000000003e-06 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.952695+0000", 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1540000000000001e-06 2024-09-14T07:09:49.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11954 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959327+0000", 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00051344100000000001, 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001308118, 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11954 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959327+0000", 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959327+0000", 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959328+0000", 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.08e-06 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959332+0000", 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3499999999999999e-06 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959343+0000", 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1246999999999999e-05 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959351+0000", 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5739999999999998e-06 2024-09-14T07:09:49.020 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11955 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959376+0000", 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00046386600000000001, 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001309951, 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11955 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959376+0000", 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959376+0000", 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959377+0000", 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0299999999999998e-07 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959380+0000", 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959387+0000", 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5170000000000002e-06 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959395+0000", 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5870000000000004e-06 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11956 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959404+0000", 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000436002, 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001337855, 2024-09-14T07:09:49.024 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11956 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959404+0000", 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959404+0000", 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959405+0000", 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3799999999999997e-07 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959407+0000", 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5459999999999998e-06 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959414+0000", 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4010000000000002e-06 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959419+0000", 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5659999999999997e-06 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11957 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959437+0000", 2024-09-14T07:09:49.027 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00040283000000000002, 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001355242, 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11957 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959437+0000", 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959437+0000", 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959438+0000", 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4499999999999997e-07 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959440+0000", 2024-09-14T07:09:49.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.373e-06 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959453+0000", 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3074999999999999e-05 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959459+0000", 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8839999999999999e-06 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11958 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959476+0000", 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00036449799999999999, 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001370142, 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11958 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959476+0000", 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959476+0000", 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959476+0000", 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8400000000000004e-07 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959490+0000", 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3811999999999999e-05 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959498+0000", 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2220000000000002e-06 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959504+0000", 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6289999999999998e-06 2024-09-14T07:09:49.033 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11959 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959515+0000", 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00032525499999999999, 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001380303, 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.034 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11959 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959515+0000", 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959515+0000", 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959516+0000", 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959518+0000", 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3379999999999999e-06 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959525+0000", 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5819999999999998e-06 2024-09-14T07:09:49.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959529+0000", 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11960 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959554+0000", 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00028659699999999998, 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001392499, 2024-09-14T07:09:49.037 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11960 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959554+0000", 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959554+0000", 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959554+0000", 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5600000000000005e-07 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959557+0000", 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4219999999999999e-06 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959564+0000", 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1810000000000004e-06 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959569+0000", 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1039999999999998e-06 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11961 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.040 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959578+0000", 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00026250299999999998, 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014198749999999999, 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11961 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959578+0000", 2024-09-14T07:09:49.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959578+0000", 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959578+0000", 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7299999999999995e-07 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959581+0000", 2024-09-14T07:09:49.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1890000000000001e-06 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959587+0000", 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7209999999999997e-06 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959593+0000", 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6130000000000003e-06 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11962 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959601+0000", 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00023908600000000001, 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001446817, 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11962 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959601+0000", 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959601+0000", 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959602+0000", 2024-09-14T07:09:49.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0800000000000004e-07 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959604+0000", 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0810000000000001e-06 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959610+0000", 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5690000000000001e-06 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959616+0000", 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1449999999999999e-06 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11963 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959624+0000", 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0002163, 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001475325, 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.047 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11963 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959624+0000", 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959624+0000", 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959625+0000", 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5499999999999998e-07 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959627+0000", 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.198e-06 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959634+0000", 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5680000000000002e-06 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959640+0000", 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2859999999999999e-06 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11964 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.050 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959647+0000", 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00019304399999999999, 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015021290000000001, 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11964 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959647+0000", 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959647+0000", 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959648+0000", 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1099999999999995e-07 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959650+0000", 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.305e-06 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959658+0000", 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4819999999999997e-06 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959663+0000", 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1139999999999997e-06 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.053 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11965 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959687+0000", 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00015350300000000001, 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015134580000000001, 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11965 2024-09-14T07:09:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959687+0000", 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959687+0000", 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959687+0000", 2024-09-14T07:09:49.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6400000000000002e-07 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959690+0000", 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.52e-06 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959698+0000", 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8490000000000006e-06 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959703+0000", 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:09:49.056 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11966 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959724+0000", 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000115828, 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015273590000000001, 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.057 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11966 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959724+0000", 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959724+0000", 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959725+0000", 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4600000000000004e-07 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959727+0000", 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3590000000000002e-06 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959736+0000", 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1019999999999997e-06 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959740+0000", 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6659999999999999e-06 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11967 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959762+0000", 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.8361000000000004e-05, 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001540965, 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11967 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959762+0000", 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959762+0000", 2024-09-14T07:09:49.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959763+0000", 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8599999999999997e-07 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959765+0000", 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8479999999999999e-06 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959769+0000", 2024-09-14T07:09:49.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6659999999999999e-06 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959772+0000", 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0479999999999999e-06 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11968 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959784+0000", 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.6623999999999999e-05, 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015712160000000001, 2024-09-14T07:09:49.063 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11968 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959784+0000", 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959784+0000", 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959784+0000", 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3700000000000001e-07 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959785+0000", 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3489999999999999e-06 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959789+0000", 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5939999999999998e-06 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959792+0000", 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7e-06 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11969 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.066 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959805+0000", 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.5683999999999999e-05, 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016016229999999999, 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11969 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959805+0000", 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959805+0000", 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959805+0000", 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2599999999999998e-07 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959807+0000", 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.584e-06 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959810+0000", 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7450000000000001e-06 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959813+0000", 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.836e-06 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11970 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959818+0000", 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 2.1756999999999999e-05, 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016384139999999999, 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11970 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959818+0000", 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959818+0000", 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959819+0000", 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959830+0000", 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1034e-05 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959834+0000", 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9709999999999998e-06 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959837+0000", 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:49.072 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11971 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959848+0000", 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999928, 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016620789999999999, 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11971 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959848+0000", 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959848+0000", 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959848+0000", 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3500000000000002e-07 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959849+0000", 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3650000000000001e-06 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959853+0000", 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.613e-06 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959856+0000", 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8619999999999999e-06 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11972 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959871+0000", 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.999969, 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00169173, 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.076 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11972 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959871+0000", 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959871+0000", 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959871+0000", 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0699999999999998e-07 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959873+0000", 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2669999999999999e-06 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959876+0000", 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5860000000000001e-06 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959879+0000", 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7939999999999998e-06 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11973 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959884+0000", 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999566, 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001772584, 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.080 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11973 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959884+0000", 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959884+0000", 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959884+0000", 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.96e-07 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959885+0000", 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.209e-06 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959888+0000", 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1379999999999999e-06 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959891+0000", 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6850000000000001e-06 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11974 3.1c 3.7e990b7c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959895+0000", 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999456, 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018126959999999999, 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.083 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11974 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959895+0000", 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959895+0000", 2024-09-14T07:09:49.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959895+0000", 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5699999999999999e-07 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959896+0000", 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1069999999999999e-06 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959899+0000", 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0929999999999999e-06 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959902+0000", 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.689e-06 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11989 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959906+0000", 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999347, 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001854442, 2024-09-14T07:09:49.086 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11989 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959906+0000", 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959906+0000", 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959906+0000", 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9999999999999998e-07 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959907+0000", 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959910+0000", 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.083e-06 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959913+0000", 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6869999999999999e-06 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11990 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.089 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959916+0000", 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999237, 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001895416, 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11990 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959916+0000", 2024-09-14T07:09:49.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959916+0000", 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959917+0000", 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1200000000000001e-06 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959919+0000", 2024-09-14T07:09:49.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.08e-06 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959923+0000", 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1359999999999999e-06 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959925+0000", 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6620000000000001e-06 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11991 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959929+0000", 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999113, 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001936035, 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11991 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959929+0000", 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959929+0000", 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959929+0000", 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9099999999999999e-07 2024-09-14T07:09:49.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959930+0000", 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.127e-06 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959933+0000", 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0900000000000001e-06 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959936+0000", 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5469999999999999e-06 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.095 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5548.0:11992 3.c 3.677d474c (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:48.959946+0000", 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998937, 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019731900000000001, 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5548", 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 11992 2024-09-14T07:09:49.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959946+0000", 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959946+0000", 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959947+0000", 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9299999999999999e-07 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959948+0000", 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.254e-06 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959952+0000", 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6059999999999999e-06 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:48.959954+0000", 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 40 2024-09-14T07:09:49.099 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:49.101 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:49.101 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:49.099+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:49.201 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:49.199+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:49.211 INFO:teuthology.orchestra.run.smithi145.stdout:3956 write 0x5272000 thru 0x527cec5 (0xaec6 bytes) 2024-09-14T07:09:49.248 INFO:teuthology.orchestra.run.smithi145.stdout:3958 write 0xa5a2200 thru 0xa5a3a5f (0x1860 bytes) 2024-09-14T07:09:49.250 INFO:teuthology.orchestra.run.smithi145.stdout:3959 read 0x80be00 thru 0x814a54 (0x8c55 bytes) 2024-09-14T07:09:49.255 INFO:teuthology.orchestra.run.smithi145.stdout:3960 punch from 0x3fdd600 to 0x3fe7c00, (0xa600 bytes) 2024-09-14T07:09:49.255 INFO:teuthology.orchestra.run.smithi145.stdout:3961 write 0x16fc600 thru 0x16fd468 (0xe69 bytes) 2024-09-14T07:09:49.256 INFO:teuthology.orchestra.run.smithi145.stdout:3963 punch from 0x342b600 to 0x343a000, (0xea00 bytes) 2024-09-14T07:09:49.302 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:49.299+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:49.307 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:49.307 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:49.321 INFO:teuthology.orchestra.run.smithi145.stdout:3965 punch from 0x38c6000 to 0x38cc200, (0x6200 bytes) 2024-09-14T07:09:49.322 INFO:teuthology.orchestra.run.smithi145.stdout:3966 read 0x2cb2000 thru 0x2cbfe11 (0xde12 bytes) 2024-09-14T07:09:49.322 INFO:teuthology.orchestra.run.smithi145.stdout:3967 punch from 0x63d6800 to 0x63df200, (0x8a00 bytes) 2024-09-14T07:09:49.327 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:49.415 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:49.415+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:09:49.490 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:49.490 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:49.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:49.490 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:49.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.490 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.658505297, 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:49.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:49.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.657730949, 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:49.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.646547951000002, 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.501 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.506 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.349619744999998, 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:49.507 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:49.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:49.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.114715920999998, 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.513 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:09:49.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.517 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.114154130000003, 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:09:49.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:09:49.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.113659562999999, 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.527 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.113042143999998, 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:09:49.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.112613885000002, 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:09:49.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.112218020999997, 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.537 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:49.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:09:49.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.111626344999998, 2024-09-14T07:09:49.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:09:49.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:09:49.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.108297231000002, 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.546 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.108256179999998, 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:09:49.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.108203920000001, 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:09:49.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:09:49.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.108165032999999, 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.560 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:09:49.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:09:49.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.108105690000002, 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:09:49.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.108051602000003, 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.569 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:09:49.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:49.574 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.108012248999998, 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.575 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:09:49.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:09:49.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.107974468999998, 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.580 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:49.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:09:49.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.585 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:49.585 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:49.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.107937765999999, 2024-09-14T07:09:49.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:09:49.589 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:49.589 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:49.589 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:49.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:49.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:49.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:09:49.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:09:49.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:49.593 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:49.596 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:49.591+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:49.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3970 write 0xc67b600 thru 0xc684a5e (0x945f bytes) 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3971 read 0xb3fd400 thru 0xb40365e (0x625f bytes) 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3974 trunc from 0xc684c00 to 0xd97c600 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3976 write 0xafe1e00 thru 0xafe9953 (0x7b54 bytes) 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3978 trunc from 0xd97c600 to 0x7d61a00 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3984 read 0x58b6200 thru 0x58bf7a3 (0x95a4 bytes) 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3985 write 0xcc66200 thru 0xcc75b90 (0xf991 bytes) 2024-09-14T07:09:49.597 INFO:teuthology.orchestra.run.smithi145.stdout:3986 punch from 0xb287c00 to 0xb295400, (0xd800 bytes) 2024-09-14T07:09:49.623 INFO:teuthology.orchestra.run.smithi145.stdout:3988 trunc from 0xcc75c00 to 0x191c00 2024-09-14T07:09:49.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:49.669 INFO:teuthology.orchestra.run.smithi145.stdout:3989 read 0x145200 thru 0x149a7d (0x487e bytes) 2024-09-14T07:09:49.702 INFO:teuthology.orchestra.run.smithi145.stdout:3990 trunc from 0x191c00 to 0xcc82200 2024-09-14T07:09:49.727 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:49.727+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:49.728 INFO:teuthology.orchestra.run.smithi145.stdout:3994 trunc from 0xcc82200 to 0xb4fd800 2024-09-14T07:09:49.761 INFO:teuthology.orchestra.run.smithi145.stdout:3999 read 0x15c400 thru 0x16734e (0xaf4f bytes) 2024-09-14T07:09:49.839 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:49.839 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:49.840 INFO:teuthology.orchestra.run.smithi145.stdout:4001 read 0x7aa3c00 thru 0x7aacd06 (0x9107 bytes) 2024-09-14T07:09:49.840 INFO:teuthology.orchestra.run.smithi145.stdout:4002 read 0x5e9f200 thru 0x5e9fe05 (0xc06 bytes) 2024-09-14T07:09:49.840 INFO:teuthology.orchestra.run.smithi145.stdout:4004 write 0x555a00 thru 0x56501d (0xf61e bytes) 2024-09-14T07:09:49.840 INFO:teuthology.orchestra.run.smithi145.stdout:4006 punch from 0xb47e400 to 0xb486800, (0x8400 bytes) 2024-09-14T07:09:49.840 INFO:teuthology.orchestra.run.smithi145.stdout:4007 write 0xbeaea00 thru 0xbeb2d32 (0x4333 bytes) 2024-09-14T07:09:49.840 INFO:teuthology.orchestra.run.smithi145.stdout:4009 read 0xbba800 thru 0xbc56ae (0xaeaf bytes) 2024-09-14T07:09:49.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:49.841 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:49.841 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:49.842 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:49.835+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:49.842 INFO:teuthology.orchestra.run.smithi145.stdout:4010 write 0xdde6e00 thru 0xddeb96a (0x4b6b bytes) 2024-09-14T07:09:49.842 INFO:teuthology.orchestra.run.smithi145.stdout:4011 punch from 0xd7a9200 to 0xd7ab600, (0x2400 bytes) 2024-09-14T07:09:49.842 INFO:teuthology.orchestra.run.smithi145.stdout:4017 write 0x7ccee00 thru 0x7cd0fec (0x21ed bytes) 2024-09-14T07:09:49.849 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:49.850 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:49.850 INFO:teuthology.orchestra.run.smithi145.stdout:4019 trunc from 0xddeba00 to 0xb252a00 2024-09-14T07:09:49.869 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:49.886 INFO:teuthology.orchestra.run.smithi145.stdout:4021 read 0x7d35a00 thru 0x7d36e60 (0x1461 bytes) 2024-09-14T07:09:49.889 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:49.899 INFO:teuthology.orchestra.run.smithi145.stdout:4022 trunc from 0xb252a00 to 0x7c3fc00 2024-09-14T07:09:49.911 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:49.930 INFO:teuthology.orchestra.run.smithi145.stdout:4026 trunc from 0x7c3fc00 to 0x4e0f000 2024-09-14T07:09:49.932 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:49.932 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:49.939 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:49.939+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:49.960 INFO:teuthology.orchestra.run.smithi145.stdout:4027 punch from 0x23cac00 to 0x23cbc00, (0x1000 bytes) 2024-09-14T07:09:49.961 INFO:teuthology.orchestra.run.smithi145.stdout:4030 punch from 0x3cc600 to 0x3cf400, (0x2e00 bytes) 2024-09-14T07:09:49.961 INFO:teuthology.orchestra.run.smithi145.stdout:4034 read 0x35bfa00 thru 0x35c0cf3 (0x12f4 bytes) 2024-09-14T07:09:49.970 INFO:teuthology.orchestra.run.smithi145.stdout:4036 trunc from 0x4e0f000 to 0xa3bea00 2024-09-14T07:09:49.970 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:49.970 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:49.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:50.010 INFO:teuthology.orchestra.run.smithi145.stdout:4037 trunc from 0xa3bea00 to 0x39a7600 2024-09-14T07:09:50.042 INFO:teuthology.orchestra.run.smithi145.stdout:4040 trunc from 0x39a7600 to 0x8a36600 2024-09-14T07:09:50.066 INFO:teuthology.orchestra.run.smithi145.stdout:4042 write 0x7c8b200 thru 0x7c8fd9e (0x4b9f bytes) 2024-09-14T07:09:50.068 INFO:teuthology.orchestra.run.smithi145.stdout:4044 trunc from 0x8a36600 to 0x8309a00 2024-09-14T07:09:50.071 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:50.071+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:50.103 INFO:teuthology.orchestra.run.smithi145.stdout:4046 trunc from 0x8309a00 to 0xaefbc00 2024-09-14T07:09:50.141 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:50.139+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:50.142 INFO:teuthology.orchestra.run.smithi145.stdout:4047 trunc from 0xaefbc00 to 0xec89000 2024-09-14T07:09:50.152 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:50.152 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:50.161 INFO:teuthology.orchestra.run.smithi145.stdout:4048 read 0xc470c00 thru 0xc47a1a1 (0x95a2 bytes) 2024-09-14T07:09:50.163 INFO:teuthology.orchestra.run.smithi145.stdout:4049 read 0x36a2a00 thru 0x36b0f01 (0xe502 bytes) 2024-09-14T07:09:50.163 INFO:teuthology.orchestra.run.smithi145.stdout:4051 write 0x43f5600 thru 0x43fd2c0 (0x7cc1 bytes) 2024-09-14T07:09:50.163 INFO:teuthology.orchestra.run.smithi145.stdout:4052 read 0x733da00 thru 0x733fc19 (0x221a bytes) 2024-09-14T07:09:50.164 INFO:teuthology.orchestra.run.smithi145.stdout:4053 read 0x2792600 thru 0x27a16c4 (0xf0c5 bytes) 2024-09-14T07:09:50.164 INFO:teuthology.orchestra.run.smithi145.stdout:4056 read 0x62fd400 thru 0x63091ed (0xbdee bytes) 2024-09-14T07:09:50.164 INFO:teuthology.orchestra.run.smithi145.stdout:4057 read 0xcf1f200 thru 0xcf2eca9 (0xfaaa bytes) 2024-09-14T07:09:50.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:50.186 INFO:teuthology.orchestra.run.smithi145.stdout:4059 trunc from 0xec89000 to 0x1765a00 2024-09-14T07:09:50.255 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:50.255+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:50.261 INFO:teuthology.orchestra.run.smithi145.stdout:4060 write 0x8fd7600 thru 0x8fe15ca (0x9fcb bytes) 2024-09-14T07:09:50.307 INFO:teuthology.orchestra.run.smithi145.stdout:4061 write 0xea89000 thru 0xea94270 (0xb271 bytes) 2024-09-14T07:09:50.310 INFO:teuthology.orchestra.run.smithi145.stdout:4062 write 0xe50ce00 thru 0xe511230 (0x4431 bytes) 2024-09-14T07:09:50.310 INFO:teuthology.orchestra.run.smithi145.stdout:4064 write 0xc7daa00 thru 0xc7e3d9c (0x939d bytes) 2024-09-14T07:09:50.310 INFO:teuthology.orchestra.run.smithi145.stdout:4065 write 0xba3d800 thru 0xba4848d (0xac8e bytes) 2024-09-14T07:09:50.310 INFO:teuthology.orchestra.run.smithi145.stdout:4066 write 0xbb56e00 thru 0xbb66695 (0xf896 bytes) 2024-09-14T07:09:50.311 INFO:teuthology.orchestra.run.smithi145.stdout:4068 write 0xea03400 thru 0xea04a32 (0x1633 bytes) 2024-09-14T07:09:50.311 INFO:teuthology.orchestra.run.smithi145.stdout:4069 write 0xac5e400 thru 0xac613da (0x2fdb bytes) 2024-09-14T07:09:50.343 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:50.343+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:50.345 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:50.345 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:50.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:50.363 INFO:teuthology.orchestra.run.smithi145.stdout:4070 write 0xed3fc00 thru 0xed45cc1 (0x60c2 bytes) 2024-09-14T07:09:50.366 INFO:teuthology.orchestra.run.smithi145.stdout:4074 write 0xb2a3000 thru 0xb2a8c44 (0x5c45 bytes) 2024-09-14T07:09:50.367 INFO:teuthology.orchestra.run.smithi145.stdout:4076 read 0xdcd4800 thru 0xdcd9597 (0x4d98 bytes) 2024-09-14T07:09:50.367 INFO:teuthology.orchestra.run.smithi145.stdout:4077 write 0xca79e00 thru 0xca7b871 (0x1a72 bytes) 2024-09-14T07:09:50.368 INFO:teuthology.orchestra.run.smithi145.stdout:4078 read 0x95a0600 thru 0x95a5639 (0x503a bytes) 2024-09-14T07:09:50.368 INFO:teuthology.orchestra.run.smithi145.stdout:4080 write 0xaa51e00 thru 0xaa53864 (0x1a65 bytes) 2024-09-14T07:09:50.369 INFO:teuthology.orchestra.run.smithi145.stdout:4082 read 0xdfc0400 thru 0xdfc430b (0x3f0c bytes) 2024-09-14T07:09:50.369 INFO:teuthology.orchestra.run.smithi145.stdout:4083 read 0xbc8c000 thru 0xbc8cd03 (0xd04 bytes) 2024-09-14T07:09:50.370 INFO:teuthology.orchestra.run.smithi145.stdout:4084 write 0x39a4c00 thru 0x39ad249 (0x864a bytes) 2024-09-14T07:09:50.370 INFO:teuthology.orchestra.run.smithi145.stdout:4086 write 0x20ff600 thru 0x210d873 (0xe274 bytes) 2024-09-14T07:09:50.371 INFO:teuthology.orchestra.run.smithi145.stdout:4087 write 0xee2800 thru 0xef25e6 (0xfde7 bytes) 2024-09-14T07:09:50.371 INFO:teuthology.orchestra.run.smithi145.stdout:4088 punch from 0x2f59e00 to 0x2f5fc00, (0x5e00 bytes) 2024-09-14T07:09:50.373 INFO:teuthology.orchestra.run.smithi145.stdout:4089 write 0x658a600 thru 0x6599441 (0xee42 bytes) 2024-09-14T07:09:50.373 INFO:teuthology.orchestra.run.smithi145.stdout:4090 read 0xca2c600 thru 0xca398f1 (0xd2f2 bytes) 2024-09-14T07:09:50.373 INFO:teuthology.orchestra.run.smithi145.stdout:4093 write 0x54d5600 thru 0x54d950e (0x3f0f bytes) 2024-09-14T07:09:50.383 INFO:teuthology.orchestra.run.smithi145.stdout:4094 trunc from 0xed45e00 to 0xa516c00 2024-09-14T07:09:50.425 INFO:teuthology.orchestra.run.smithi145.stdout:4095 read 0x6d2ea00 thru 0x6d33d2e (0x532f bytes) 2024-09-14T07:09:50.436 INFO:teuthology.orchestra.run.smithi145.stdout:4097 trunc from 0xa516c00 to 0xea11a00 2024-09-14T07:09:50.444 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:50.443+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:50.491 INFO:teuthology.orchestra.run.smithi145.stdout:4098 trunc from 0xea11a00 to 0x202be00 2024-09-14T07:09:50.528 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:50.529 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:50.531 INFO:teuthology.orchestra.run.smithi145.stdout:4099 trunc from 0x202be00 to 0x32b7e00 2024-09-14T07:09:50.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:50.545 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:50.543+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:50.564 INFO:teuthology.orchestra.run.smithi145.stdout:4102 read 0x2247800 thru 0x224d496 (0x5c97 bytes) 2024-09-14T07:09:50.567 INFO:teuthology.orchestra.run.smithi145.stdout:4104 trunc from 0x32b7e00 to 0x3316e00 2024-09-14T07:09:50.596 INFO:teuthology.orchestra.run.smithi145.stdout:4105 read 0xa70e00 thru 0xa79548 (0x8749 bytes) 2024-09-14T07:09:50.644 INFO:teuthology.orchestra.run.smithi145.stdout:4106 write 0xa434a00 thru 0xa43a4ed (0x5aee bytes) 2024-09-14T07:09:50.645 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:50.643+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:50.646 INFO:teuthology.orchestra.run.smithi145.stdout:4107 read 0x1e58400 thru 0x1e5a4c1 (0x20c2 bytes) 2024-09-14T07:09:50.647 INFO:teuthology.orchestra.run.smithi145.stdout:4108 punch from 0x89d2400 to 0x89d6200, (0x3e00 bytes) 2024-09-14T07:09:50.648 INFO:teuthology.orchestra.run.smithi145.stdout:4111 read 0x6812000 thru 0x681fe0b (0xde0c bytes) 2024-09-14T07:09:50.649 INFO:teuthology.orchestra.run.smithi145.stdout:4112 punch from 0xa09c00 to 0xa0f000, (0x5400 bytes) 2024-09-14T07:09:50.649 INFO:teuthology.orchestra.run.smithi145.stdout:4113 read 0x42cf600 thru 0x42de5a5 (0xefa6 bytes) 2024-09-14T07:09:50.649 INFO:teuthology.orchestra.run.smithi145.stdout:4114 write 0x4277600 thru 0x427ac68 (0x3669 bytes) 2024-09-14T07:09:50.649 INFO:teuthology.orchestra.run.smithi145.stdout:4115 punch from 0x39e5a00 to 0x39f0800, (0xae00 bytes) 2024-09-14T07:09:50.649 INFO:teuthology.orchestra.run.smithi145.stdout:4117 read 0x961e400 thru 0x9628a8b (0xa68c bytes) 2024-09-14T07:09:50.650 INFO:teuthology.orchestra.run.smithi145.stdout:4119 write 0x5cc7000 thru 0x5cc940c (0x240d bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4123 read 0x44d3c00 thru 0x44d4dde (0x11df bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4124 write 0x5124400 thru 0x5125c29 (0x182a bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4125 write 0x553a000 thru 0x553a983 (0x984 bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4127 read 0x4a56400 thru 0x4a5c09d (0x5c9e bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4128 read 0x4f38400 thru 0x4f3e7fc (0x63fd bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4129 read 0x5024e00 thru 0x5027a5b (0x2c5c bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4130 write 0x666d800 thru 0x667bf16 (0xe717 bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4131 read 0xe30200 thru 0xe31ccc (0x1acd bytes) 2024-09-14T07:09:50.651 INFO:teuthology.orchestra.run.smithi145.stdout:4132 write 0x861b800 thru 0x8622a80 (0x7281 bytes) 2024-09-14T07:09:50.652 INFO:teuthology.orchestra.run.smithi145.stdout:4133 punch from 0xa2d0e00 to 0xa2dd000, (0xc200 bytes) 2024-09-14T07:09:50.655 INFO:teuthology.orchestra.run.smithi145.stdout:4134 trunc from 0xa43a600 to 0xbf31c00 2024-09-14T07:09:50.681 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:50.681 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:50.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:50.700 INFO:teuthology.orchestra.run.smithi145.stdout:4135 punch from 0x6806400 to 0x6814c00, (0xe800 bytes) 2024-09-14T07:09:50.702 INFO:teuthology.orchestra.run.smithi145.stdout:4136 write 0x158d800 thru 0x15932b0 (0x5ab1 bytes) 2024-09-14T07:09:50.702 INFO:teuthology.orchestra.run.smithi145.stdout:4137 punch from 0x3b4d800 to 0x3b54400, (0x6c00 bytes) 2024-09-14T07:09:50.702 INFO:teuthology.orchestra.run.smithi145.stdout:4138 read 0x4f59a00 thru 0x4f5d362 (0x3963 bytes) 2024-09-14T07:09:50.703 INFO:teuthology.orchestra.run.smithi145.stdout:4140 read 0x4e26a00 thru 0x4e31969 (0xaf6a bytes) 2024-09-14T07:09:50.733 INFO:teuthology.orchestra.run.smithi145.stdout:4142 write 0xc86be00 thru 0xc87812e (0xc32f bytes) 2024-09-14T07:09:50.735 INFO:teuthology.orchestra.run.smithi145.stdout:4143 write 0xb104e00 thru 0xb10b87d (0x6a7e bytes) 2024-09-14T07:09:50.760 INFO:teuthology.orchestra.run.smithi145.stdout:4144 trunc from 0xc878200 to 0x31b2600 2024-09-14T07:09:50.783 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:50.783+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:50.815 INFO:teuthology.orchestra.run.smithi145.stdout:4145 punch from 0x2867800 to 0x2871200, (0x9a00 bytes) 2024-09-14T07:09:50.817 INFO:teuthology.orchestra.run.smithi145.stdout:4146 write 0x19a5200 thru 0x19ab065 (0x5e66 bytes) 2024-09-14T07:09:50.818 INFO:teuthology.orchestra.run.smithi145.stdout:4148 read 0x960000 thru 0x96b3d5 (0xb3d6 bytes) 2024-09-14T07:09:50.831 INFO:teuthology.orchestra.run.smithi145.stdout:4149 trunc from 0x31b2600 to 0x72a5c00 2024-09-14T07:09:50.847 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:50.843+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:50.858 INFO:teuthology.orchestra.run.smithi145.stdout:4156 read 0x26cbe00 thru 0x26d2ed2 (0x70d3 bytes) 2024-09-14T07:09:50.863 INFO:teuthology.orchestra.run.smithi145.stdout:4157 read 0x68a7400 thru 0x68ad411 (0x6012 bytes) 2024-09-14T07:09:50.865 INFO:teuthology.orchestra.run.smithi145.stdout:4158 trunc from 0x72a5c00 to 0x6ae3400 2024-09-14T07:09:50.873 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:50.873 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:50.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:50.899 INFO:teuthology.orchestra.run.smithi145.stdout:4163 write 0x4151200 thru 0x415815b (0x6f5c bytes) 2024-09-14T07:09:50.901 INFO:teuthology.orchestra.run.smithi145.stdout:4164 write 0x36bfa00 thru 0x36c07e9 (0xdea bytes) 2024-09-14T07:09:50.902 INFO:teuthology.orchestra.run.smithi145.stdout:4165 punch from 0x1bde600 to 0x1bee000, (0xfa00 bytes) 2024-09-14T07:09:50.940 INFO:teuthology.orchestra.run.smithi145.stdout:4166 write 0xce54c00 thru 0xce599c2 (0x4dc3 bytes) 2024-09-14T07:09:50.942 INFO:teuthology.orchestra.run.smithi145.stdout:4169 trunc from 0xce59a00 to 0xd494400 2024-09-14T07:09:50.975 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:50.975+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:50.976 INFO:teuthology.orchestra.run.smithi145.stdout:4170 trunc from 0xd494400 to 0x958ea00 2024-09-14T07:09:51.010 INFO:teuthology.orchestra.run.smithi145.stdout:4172 trunc from 0x958ea00 to 0x5f4d200 2024-09-14T07:09:51.048 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:51.047+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:51.054 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:51.054 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:51.061 INFO:teuthology.orchestra.run.smithi145.stdout:4174 write 0x72ed800 thru 0x72f8eed (0xb6ee bytes) 2024-09-14T07:09:51.061 INFO:teuthology.orchestra.run.smithi145.stdout:4176 clone 29 order 19 su 131072 sc 14 2024-09-14T07:09:51.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:51.149 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:51.147+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:51.242 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:51.243 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:51.250 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:51.247+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:51.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:51.351 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:51.347+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:51.386 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 2, 4, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:09:51.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T07:09:51.389 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:51.390 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:51.403 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:51.452 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:51.451+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:51.533 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:51.533 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:51.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:51.592 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:51.591+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:51.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:09:51.633 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone28 from 0x72f9000 (overlap 0x191c00) to 0x0 2024-09-14T07:09:51.654 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:51.651+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:51.723 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:51.723 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:51.738 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:51.784 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:51.783+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:51.856 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:51.855+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:51.909 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:51.910 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:51.924 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:51.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:51.967 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:51.967+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:52.044 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:52.044 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:52.057 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:52.055+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:52.058 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:52.101 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T07:09:52.108 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:52.126 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:52.126 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:52.142 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:52.142 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:52.158 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:52.155+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:52.161 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:52.178 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:52.179 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:52.198 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:52.252 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:52.252 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:52.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.954051739999997, 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:52.253 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:09:52.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:09:52.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 82.094576153000006, 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:09:52.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:52.263 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 82.094473867999994, 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:09:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.395422026999995, 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:09:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910500835000001, 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:09:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910465615, 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:09:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910410466000002, 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:09:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:09:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910376397, 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910333051000002, 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:09:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910299528000003, 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910252278000002, 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910138029999999, 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:09:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:09:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.910028335, 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.909962192000002, 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.909845687999997, 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:09:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:09:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.909780032, 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:09:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.909720712000002, 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.909579844, 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.909164736999998, 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.909096998000003, 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:09:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:09:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.342 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:52.344 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:52.344 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:52.343+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:52.444 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:52.443+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:52.482 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T07:09:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:09:52.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3771 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541804+0000", 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0040720660000000001, 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0040877520000000001, 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3771 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541804+0000", 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541804+0000", 2024-09-14T07:09:52.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541805+0000", 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8500000000000001e-07 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541807+0000", 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1330000000000002e-06 2024-09-14T07:09:52.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541813+0000", 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541818+0000", 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9679999999999997e-06 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3803 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542552+0000", 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0033247569999999998, 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033810279999999999, 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3803 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.557 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542552+0000", 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542552+0000", 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542552+0000", 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:09:52.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542554+0000", 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.956e-06 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542560+0000", 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7389999999999996e-06 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542564+0000", 2024-09-14T07:09:52.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3100000000000002e-06 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3835 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543282+0000", 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025945159999999998, 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002677282, 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3835 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543282+0000", 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543282+0000", 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543282+0000", 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7399999999999998e-07 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543284+0000", 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9819999999999998e-06 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543292+0000", 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5009999999999998e-06 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543297+0000", 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9300000000000002e-06 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3867 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545053+0000", 2024-09-14T07:09:52.563 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00082340199999999999, 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00093199600000000004, 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3867 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545053+0000", 2024-09-14T07:09:52.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545053+0000", 2024-09-14T07:09:52.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545053+0000", 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545055+0000", 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.378e-06 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545059+0000", 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.216e-06 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545063+0000", 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2579999999999997e-06 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3772 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541824+0000", 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0040518949999999998, 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041894640000000004, 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3772 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.568 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541824+0000", 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541824+0000", 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541825+0000", 2024-09-14T07:09:52.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5999999999999999e-07 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541827+0000", 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.221e-06 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541833+0000", 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1310000000000003e-06 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541838+0000", 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6040000000000003e-06 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3804 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542570+0000", 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0033059600000000001, 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034704750000000002, 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3804 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542570+0000", 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542570+0000", 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542571+0000", 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542573+0000", 2024-09-14T07:09:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2280000000000001e-06 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542579+0000", 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8409999999999996e-06 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542583+0000", 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4730000000000002e-06 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3836 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543309+0000", 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025669180000000001, 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027636940000000001, 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3836 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543309+0000", 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543309+0000", 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543310+0000", 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543312+0000", 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0870000000000001e-06 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543317+0000", 2024-09-14T07:09:52.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9459999999999997e-06 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543320+0000", 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1039999999999998e-06 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3868 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545070+0000", 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00080651400000000004, 2024-09-14T07:09:52.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001032222, 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3868 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545070+0000", 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545070+0000", 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545070+0000", 2024-09-14T07:09:52.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.32e-07 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545072+0000", 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1339999999999998e-06 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545078+0000", 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9920000000000004e-06 2024-09-14T07:09:52.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545084+0000", 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1229999999999999e-06 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3773 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541844+0000", 2024-09-14T07:09:52.582 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0040321360000000004, 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042889790000000001, 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3773 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541844+0000", 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541844+0000", 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541845+0000", 2024-09-14T07:09:52.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7399999999999998e-07 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541847+0000", 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1059999999999998e-06 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541853+0000", 2024-09-14T07:09:52.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9089999999999997e-06 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541857+0000", 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6619999999999996e-06 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3805 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542590+0000", 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0032864510000000001, 2024-09-14T07:09:52.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003570237, 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3805 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542590+0000", 2024-09-14T07:09:52.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542590+0000", 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542590+0000", 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1399999999999997e-07 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542592+0000", 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.069e-06 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542598+0000", 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542604+0000", 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6359999999999999e-06 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.589 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3837 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543325+0000", 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002551818, 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028674659999999999, 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3837 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543325+0000", 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543325+0000", 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543325+0000", 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.89e-07 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543326+0000", 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.302e-06 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543330+0000", 2024-09-14T07:09:52.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3280000000000002e-06 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543332+0000", 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6249999999999999e-06 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3869 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545098+0000", 2024-09-14T07:09:52.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00077828299999999995, 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011195770000000001, 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3869 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545098+0000", 2024-09-14T07:09:52.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545098+0000", 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545099+0000", 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545101+0000", 2024-09-14T07:09:52.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6240000000000002e-06 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545109+0000", 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9170000000000006e-06 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545114+0000", 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0710000000000002e-06 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3774 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.597 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541875+0000", 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0040012590000000001, 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043761989999999999, 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3774 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541875+0000", 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541875+0000", 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541876+0000", 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2e-07 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541878+0000", 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6259999999999999e-06 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541885+0000", 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999997e-06 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541891+0000", 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1850000000000001e-06 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3806 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542612+0000", 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003264674, 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036695120000000002, 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.601 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3806 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542612+0000", 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542612+0000", 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542612+0000", 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4700000000000002e-07 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542614+0000", 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.063e-06 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542620+0000", 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.699e-06 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542625+0000", 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.668e-06 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3838 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543336+0000", 2024-09-14T07:09:52.604 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025406389999999999, 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002970964, 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3838 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543336+0000", 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543336+0000", 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543336+0000", 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4499999999999998e-07 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543337+0000", 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1909999999999999e-06 2024-09-14T07:09:52.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543340+0000", 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2179999999999999e-06 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543343+0000", 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8420000000000001e-06 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3870 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545125+0000", 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00075118199999999998, 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001211251, 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3870 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.608 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545125+0000", 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545125+0000", 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545126+0000", 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545128+0000", 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.216e-06 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545135+0000", 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8000000000000001e-06 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545141+0000", 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5150000000000003e-06 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3775 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541916+0000", 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0039599279999999997, 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044512550000000003, 2024-09-14T07:09:52.611 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3775 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541916+0000", 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541916+0000", 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541917+0000", 2024-09-14T07:09:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0399999999999996e-07 2024-09-14T07:09:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541919+0000", 2024-09-14T07:09:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.525e-06 2024-09-14T07:09:52.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541926+0000", 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3620000000000002e-06 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541931+0000", 2024-09-14T07:09:52.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3807 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542631+0000", 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003245272, 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0037687110000000001, 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3807 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542631+0000", 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542631+0000", 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542632+0000", 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9999999999999998e-07 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542634+0000", 2024-09-14T07:09:52.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0669999999999999e-06 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542640+0000", 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542645+0000", 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.746e-06 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3839 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543347+0000", 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025297620000000001, 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030781070000000001, 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3839 2024-09-14T07:09:52.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543347+0000", 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543347+0000", 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543347+0000", 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3699999999999999e-07 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543348+0000", 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.271e-06 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543351+0000", 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1659999999999998e-06 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543354+0000", 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.632e-06 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3871 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545149+0000", 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000727434, 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001305872, 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.623 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3871 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545149+0000", 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545149+0000", 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545149+0000", 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8500000000000002e-07 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545151+0000", 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.581e-06 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545165+0000", 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4134999999999999e-05 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545171+0000", 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1689999999999997e-06 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.626 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3776 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541939+0000", 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0039378540000000002, 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045455219999999998, 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.627 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3776 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541939+0000", 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541939+0000", 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541939+0000", 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541941+0000", 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1840000000000002e-06 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541947+0000", 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1020000000000002e-06 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541952+0000", 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5210000000000004e-06 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.630 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3808 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542652+0000", 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0032247650000000001, 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003857114, 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3808 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542652+0000", 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542652+0000", 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542652+0000", 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8099999999999998e-07 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542654+0000", 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8190000000000001e-06 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542670+0000", 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6167999999999998e-05 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542676+0000", 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1419999999999998e-06 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3840 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543357+0000", 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025189800000000001, 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031793920000000001, 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3840 2024-09-14T07:09:52.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543357+0000", 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543357+0000", 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543358+0000", 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4499999999999998e-07 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543359+0000", 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.296e-06 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543362+0000", 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0929999999999999e-06 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543365+0000", 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6220000000000001e-06 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3872 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545181+0000", 2024-09-14T07:09:52.638 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000695373, 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013839600000000001, 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3872 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545181+0000", 2024-09-14T07:09:52.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545181+0000", 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545181+0000", 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8699999999999995e-07 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545184+0000", 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6079999999999998e-06 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545191+0000", 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0010000000000004e-06 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545196+0000", 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9590000000000003e-06 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.641 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3777 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541959+0000", 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0039178149999999998, 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046322730000000001, 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.642 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3777 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541959+0000", 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541959+0000", 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541959+0000", 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9499999999999998e-07 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541961+0000", 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0090000000000002e-06 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541967+0000", 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1029999999999998e-06 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541972+0000", 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6070000000000001e-06 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3809 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542684+0000", 2024-09-14T07:09:52.645 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003192163, 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039310359999999997, 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3809 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542684+0000", 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542684+0000", 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542685+0000", 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7800000000000002e-07 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542700+0000", 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5716999999999999e-05 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542714+0000", 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3468e-05 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542719+0000", 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1529999999999996e-06 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3841 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.649 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543368+0000", 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025084030000000002, 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032713400000000002, 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3841 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.650 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543368+0000", 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543368+0000", 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543368+0000", 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4200000000000002e-07 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543369+0000", 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.268e-06 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543374+0000", 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0820000000000001e-06 2024-09-14T07:09:52.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543377+0000", 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2770000000000001e-06 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3880 3.c 3.deb9dac (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545221+0000", 2024-09-14T07:09:52.653 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00065523399999999998, 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014473730000000001, 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3880 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545221+0000", 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545221+0000", 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545222+0000", 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7800000000000001e-07 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545224+0000", 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4499999999999998e-06 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545230+0000", 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1349999999999997e-06 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545235+0000", 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.843e-06 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3778 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541978+0000", 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0038982180000000002, 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047163209999999999, 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3778 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541978+0000", 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541978+0000", 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541979+0000", 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541981+0000", 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.204e-06 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541987+0000", 2024-09-14T07:09:52.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9780000000000002e-06 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541991+0000", 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6689999999999997e-06 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3810 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542742+0000", 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003134043, 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039805550000000002, 2024-09-14T07:09:52.660 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3810 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542742+0000", 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542742+0000", 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542743+0000", 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8299999999999997e-07 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542745+0000", 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542752+0000", 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1639999999999998e-06 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542757+0000", 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0270000000000003e-06 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.663 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3842 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543380+0000", 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0024960009999999999, 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033670900000000001, 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3842 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543380+0000", 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543380+0000", 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543381+0000", 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:52.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543382+0000", 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543385+0000", 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0910000000000001e-06 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543388+0000", 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:09:52.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3779 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.541998+0000", 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003878542, 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047790389999999997, 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3779 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541998+0000", 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541998+0000", 2024-09-14T07:09:52.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.541998+0000", 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2300000000000002e-07 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542001+0000", 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3590000000000002e-06 2024-09-14T07:09:52.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542007+0000", 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9159999999999998e-06 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542011+0000", 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8330000000000001e-06 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3811 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542764+0000", 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0031122200000000002, 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0040421679999999996, 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3811 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.671 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542764+0000", 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542764+0000", 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542765+0000", 2024-09-14T07:09:52.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8599999999999998e-07 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542767+0000", 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.413e-06 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542773+0000", 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1809999999999998e-06 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542778+0000", 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5159999999999996e-06 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3843 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543391+0000", 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0024853520000000001, 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034403519999999998, 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.674 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3843 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543391+0000", 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543391+0000", 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543391+0000", 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543393+0000", 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3510000000000001e-06 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543396+0000", 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1269999999999999e-06 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543398+0000", 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5280000000000002e-06 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3887 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545614+0000", 2024-09-14T07:09:52.677 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00026232700000000002, 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001255501, 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3887 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545614+0000", 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545614+0000", 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545615+0000", 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4700000000000001e-07 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545619+0000", 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3460000000000004e-06 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545627+0000", 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939999999999995e-06 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545633+0000", 2024-09-14T07:09:52.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0900000000000001e-06 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3780 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542018+0000", 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0038588160000000002, 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048792009999999997, 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3780 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542018+0000", 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542018+0000", 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542018+0000", 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2e-07 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542020+0000", 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0200000000000001e-06 2024-09-14T07:09:52.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542026+0000", 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4949999999999996e-06 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542031+0000", 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5700000000000003e-06 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3812 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542784+0000", 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0030919849999999998, 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041412039999999999, 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3812 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542784+0000", 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542784+0000", 2024-09-14T07:09:52.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542785+0000", 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542787+0000", 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.052e-06 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542793+0000", 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542798+0000", 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5379999999999996e-06 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.688 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3844 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543402+0000", 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0024747749999999998, 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035493170000000002, 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3844 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543402+0000", 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543402+0000", 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543402+0000", 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3699999999999999e-07 2024-09-14T07:09:52.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543403+0000", 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3239999999999999e-06 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543407+0000", 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2250000000000002e-06 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543412+0000", 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3329999999999998e-06 2024-09-14T07:09:52.691 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3888 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545643+0000", 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00023371899999999999, 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013372309999999999, 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3888 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545643+0000", 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545643+0000", 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545643+0000", 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1800000000000001e-07 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545645+0000", 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.739e-06 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545649+0000", 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8469999999999998e-06 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545652+0000", 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1149999999999998e-06 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3781 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542038+0000", 2024-09-14T07:09:52.695 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0038385170000000001, 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0049732170000000003, 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3781 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542038+0000", 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542038+0000", 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542038+0000", 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542040+0000", 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9599999999999999e-06 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542046+0000", 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1e-06 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542051+0000", 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3813 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542804+0000", 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0030720959999999999, 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042344590000000003, 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3813 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.700 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542804+0000", 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542804+0000", 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542805+0000", 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400000000000001e-07 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542807+0000", 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0360000000000001e-06 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542812+0000", 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6280000000000002e-06 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542817+0000", 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7859999999999996e-06 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3845 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543419+0000", 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002457384, 2024-09-14T07:09:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036603120000000002, 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3845 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543419+0000", 2024-09-14T07:09:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543419+0000", 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543420+0000", 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1799999999999995e-07 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543427+0000", 2024-09-14T07:09:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4250000000000001e-06 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543433+0000", 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3049999999999998e-06 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543438+0000", 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7999999999999998e-06 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3889 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545657+0000", 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00021957300000000001, 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014480879999999999, 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3889 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545657+0000", 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545657+0000", 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545657+0000", 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1399999999999997e-07 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545658+0000", 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.249e-06 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545662+0000", 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2600000000000001e-06 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545665+0000", 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9399999999999998e-06 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3782 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542058+0000", 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0038182060000000002, 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050780479999999999, 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3782 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542058+0000", 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542058+0000", 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542059+0000", 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542060+0000", 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9290000000000001e-06 2024-09-14T07:09:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542066+0000", 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6049999999999997e-06 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542071+0000", 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3940000000000004e-06 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3814 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542824+0000", 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0030527779999999999, 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043416100000000001, 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3814 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542824+0000", 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542824+0000", 2024-09-14T07:09:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542824+0000", 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2300000000000002e-07 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542826+0000", 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0760000000000001e-06 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542832+0000", 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9730000000000002e-06 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542838+0000", 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.84e-06 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3846 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543445+0000", 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0024311409999999999, 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0037572090000000001, 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3846 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543445+0000", 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543445+0000", 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543446+0000", 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:09:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543448+0000", 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.057e-06 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543455+0000", 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3479999999999998e-06 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543460+0000", 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8539999999999996e-06 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3890 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545668+0000", 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00020809999999999999, 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015591299999999999, 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3890 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545668+0000", 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545668+0000", 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545669+0000", 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7099999999999998e-07 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545670+0000", 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2410000000000001e-06 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545673+0000", 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1489999999999998e-06 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545676+0000", 2024-09-14T07:09:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6790000000000001e-06 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3783 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542096+0000", 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0037807969999999998, 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051635190000000001, 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3783 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542096+0000", 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542096+0000", 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542096+0000", 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8999999999999997e-07 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542099+0000", 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4499999999999998e-06 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542105+0000", 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.455e-06 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542110+0000", 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8239999999999999e-06 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3815 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542845+0000", 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0030318709999999998, 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044390460000000003, 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3815 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542845+0000", 2024-09-14T07:09:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542845+0000", 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542845+0000", 2024-09-14T07:09:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0600000000000001e-07 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542847+0000", 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0090000000000002e-06 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542853+0000", 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4969999999999998e-06 2024-09-14T07:09:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542858+0000", 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9169999999999997e-06 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3847 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543467+0000", 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0024098800000000001, 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0038477939999999999, 2024-09-14T07:09:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3847 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543467+0000", 2024-09-14T07:09:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543467+0000", 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543467+0000", 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6400000000000003e-07 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543469+0000", 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543476+0000", 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5650000000000004e-06 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543482+0000", 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.614e-06 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3891 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545679+0000", 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000196985, 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016666770000000001, 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3891 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545679+0000", 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545679+0000", 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545680+0000", 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545684+0000", 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.087e-06 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545687+0000", 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3349999999999998e-06 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545690+0000", 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.762e-06 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3784 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542118+0000", 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0037587269999999999, 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052579020000000001, 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3784 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542118+0000", 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542118+0000", 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542118+0000", 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6400000000000003e-07 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542120+0000", 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2050000000000001e-06 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542126+0000", 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8379999999999998e-06 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542131+0000", 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5929999999999999e-06 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3816 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542865+0000", 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0030113399999999999, 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045396689999999996, 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3816 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542865+0000", 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542865+0000", 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542866+0000", 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5400000000000001e-07 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542868+0000", 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9709999999999998e-06 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542874+0000", 2024-09-14T07:09:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542878+0000", 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6149999999999999e-06 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3848 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543489+0000", 2024-09-14T07:09:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0023875419999999999, 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003944347, 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3848 2024-09-14T07:09:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543489+0000", 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543489+0000", 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543489+0000", 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4000000000000002e-07 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543491+0000", 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.086e-06 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543498+0000", 2024-09-14T07:09:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543503+0000", 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5449999999999997e-06 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3892 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545694+0000", 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000182609, 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017661650000000001, 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3892 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545694+0000", 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545694+0000", 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545694+0000", 2024-09-14T07:09:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545695+0000", 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.22e-06 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545698+0000", 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0929999999999999e-06 2024-09-14T07:09:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545701+0000", 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6589999999999999e-06 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3785 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542137+0000", 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0037391109999999998, 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0053546100000000001, 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3785 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542137+0000", 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542137+0000", 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542138+0000", 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542151+0000", 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3522e-05 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542158+0000", 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5080000000000002e-06 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542163+0000", 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3469999999999998e-06 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3817 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542884+0000", 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002992013, 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046383010000000001, 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3817 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542884+0000", 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542884+0000", 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542885+0000", 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8599999999999998e-07 2024-09-14T07:09:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542887+0000", 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9709999999999998e-06 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542893+0000", 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0290000000000002e-06 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542898+0000", 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9339999999999997e-06 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3849 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543510+0000", 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002366862, 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0040417760000000004, 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3849 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543510+0000", 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543510+0000", 2024-09-14T07:09:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543510+0000", 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5699999999999998e-07 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543512+0000", 2024-09-14T07:09:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1320000000000001e-06 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543518+0000", 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0929999999999999e-06 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543524+0000", 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8680000000000004e-06 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3893 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545704+0000", 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00017201599999999999, 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001874012, 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3893 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545704+0000", 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545704+0000", 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545705+0000", 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545706+0000", 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1739999999999999e-06 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545709+0000", 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5949999999999999e-06 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545712+0000", 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8439999999999998e-06 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3786 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542173+0000", 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0037036579999999999, 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0054310670000000004, 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3786 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542173+0000", 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542173+0000", 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542173+0000", 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542175+0000", 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.216e-06 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542181+0000", 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0290000000000002e-06 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542186+0000", 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7940000000000002e-06 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3818 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542923+0000", 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0029536720000000001, 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047070039999999999, 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3818 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542923+0000", 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542923+0000", 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542923+0000", 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999999e-07 2024-09-14T07:09:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542926+0000", 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5749999999999999e-06 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542932+0000", 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5880000000000003e-06 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542937+0000", 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8269999999999997e-06 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3850 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543531+0000", 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0023458559999999999, 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041232960000000003, 2024-09-14T07:09:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3850 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543531+0000", 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543531+0000", 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543531+0000", 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543533+0000", 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9649999999999998e-06 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543539+0000", 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6590000000000004e-06 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543543+0000", 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4649999999999996e-06 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3894 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545716+0000", 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00015990500000000001, 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019627630000000002, 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3894 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545716+0000", 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545716+0000", 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545717+0000", 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7700000000000001e-07 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545718+0000", 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.195e-06 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545721+0000", 2024-09-14T07:09:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.01e-06 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545724+0000", 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3787 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542193+0000", 2024-09-14T07:09:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003683511, 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0055116369999999998, 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3787 2024-09-14T07:09:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542193+0000", 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542193+0000", 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542193+0000", 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.75e-07 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542195+0000", 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542201+0000", 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5659999999999997e-06 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542205+0000", 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6040000000000003e-06 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3819 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542946+0000", 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0029305680000000001, 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047846199999999998, 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3819 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542946+0000", 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542946+0000", 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542946+0000", 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542948+0000", 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542954+0000", 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7470000000000002e-06 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542959+0000", 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3851 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543550+0000", 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0023264929999999998, 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042048399999999996, 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3851 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543550+0000", 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543550+0000", 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543550+0000", 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3799999999999998e-07 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543552+0000", 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9470000000000002e-06 2024-09-14T07:09:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543559+0000", 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.241e-06 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543563+0000", 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4610000000000001e-06 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3895 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545727+0000", 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00014906199999999999, 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020530100000000001, 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3895 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545727+0000", 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545727+0000", 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545728+0000", 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7700000000000001e-07 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545729+0000", 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.218e-06 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545733+0000", 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3189999999999996e-06 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545738+0000", 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5879999999999999e-06 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3788 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542213+0000", 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0036632230000000002, 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0055923099999999996, 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3788 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542213+0000", 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542213+0000", 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542214+0000", 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9100000000000004e-07 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542216+0000", 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0370000000000001e-06 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542222+0000", 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9800000000000003e-06 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542226+0000", 2024-09-14T07:09:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4889999999999997e-06 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3820 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542966+0000", 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002910059, 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0049212199999999996, 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3820 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542966+0000", 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542966+0000", 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542967+0000", 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1800000000000001e-07 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542969+0000", 2024-09-14T07:09:52.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9630000000000001e-06 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542975+0000", 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0460000000000002e-06 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542980+0000", 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4840000000000003e-06 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.808 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3852 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543571+0000", 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0023054149999999999, 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043431759999999998, 2024-09-14T07:09:52.809 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.810 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.810 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.810 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3852 2024-09-14T07:09:52.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.811 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543571+0000", 2024-09-14T07:09:52.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543571+0000", 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543571+0000", 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6499999999999999e-07 2024-09-14T07:09:52.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543574+0000", 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0849999999999999e-06 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543580+0000", 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0360000000000003e-06 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543585+0000", 2024-09-14T07:09:52.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0270000000000003e-06 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3896 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545758+0000", 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000118366, 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021803479999999999, 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.814 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3896 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545758+0000", 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545758+0000", 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545759+0000", 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7100000000000001e-07 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545761+0000", 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545768+0000", 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8430000000000004e-06 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545773+0000", 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0340000000000004e-06 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3789 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542233+0000", 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0036434729999999999, 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057325420000000002, 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.818 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3789 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542233+0000", 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542233+0000", 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542233+0000", 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.06e-07 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542235+0000", 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0190000000000001e-06 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542241+0000", 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7860000000000002e-06 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542246+0000", 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3821 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542987+0000", 2024-09-14T07:09:52.822 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0028889929999999999, 2024-09-14T07:09:52.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050034529999999997, 2024-09-14T07:09:52.823 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.823 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.823 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3821 2024-09-14T07:09:52.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542987+0000", 2024-09-14T07:09:52.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542987+0000", 2024-09-14T07:09:52.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542988+0000", 2024-09-14T07:09:52.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:09:52.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543001+0000", 2024-09-14T07:09:52.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3572e-05 2024-09-14T07:09:52.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543008+0000", 2024-09-14T07:09:52.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.72e-06 2024-09-14T07:09:52.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543013+0000", 2024-09-14T07:09:52.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:09:52.830 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.830 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.830 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.830 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3853 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.831 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543591+0000", 2024-09-14T07:09:52.831 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002284927, 2024-09-14T07:09:52.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044260369999999999, 2024-09-14T07:09:52.831 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3853 2024-09-14T07:09:52.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.833 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543591+0000", 2024-09-14T07:09:52.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543591+0000", 2024-09-14T07:09:52.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543592+0000", 2024-09-14T07:09:52.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7100000000000002e-07 2024-09-14T07:09:52.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543594+0000", 2024-09-14T07:09:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2539999999999999e-06 2024-09-14T07:09:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543600+0000", 2024-09-14T07:09:52.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.772e-06 2024-09-14T07:09:52.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543607+0000", 2024-09-14T07:09:52.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0520000000000004e-06 2024-09-14T07:09:52.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.838 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3897 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.839 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545781+0000", 2024-09-14T07:09:52.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.5610000000000001e-05, 2024-09-14T07:09:52.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022616110000000002, 2024-09-14T07:09:52.839 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.841 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3897 2024-09-14T07:09:52.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.841 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545781+0000", 2024-09-14T07:09:52.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545781+0000", 2024-09-14T07:09:52.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545781+0000", 2024-09-14T07:09:52.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6899999999999998e-07 2024-09-14T07:09:52.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545783+0000", 2024-09-14T07:09:52.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0420000000000001e-06 2024-09-14T07:09:52.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545789+0000", 2024-09-14T07:09:52.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1720000000000004e-06 2024-09-14T07:09:52.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545794+0000", 2024-09-14T07:09:52.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5850000000000001e-06 2024-09-14T07:09:52.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.846 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.847 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3790 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.847 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542272+0000", 2024-09-14T07:09:52.847 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0036043350000000002, 2024-09-14T07:09:52.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.005797829, 2024-09-14T07:09:52.848 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.848 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3790 2024-09-14T07:09:52.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.849 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542272+0000", 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542272+0000", 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542273+0000", 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7300000000000001e-07 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542275+0000", 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.464e-06 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542282+0000", 2024-09-14T07:09:52.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0790000000000003e-06 2024-09-14T07:09:52.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542287+0000", 2024-09-14T07:09:52.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8840000000000002e-06 2024-09-14T07:09:52.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.854 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3822 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.854 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543022+0000", 2024-09-14T07:09:52.854 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0028541640000000002, 2024-09-14T07:09:52.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050720749999999997, 2024-09-14T07:09:52.854 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.855 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.855 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.855 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.855 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.855 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3822 2024-09-14T07:09:52.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.856 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543022+0000", 2024-09-14T07:09:52.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543022+0000", 2024-09-14T07:09:52.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543023+0000", 2024-09-14T07:09:52.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543025+0000", 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0590000000000001e-06 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543038+0000", 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2935000000000001e-05 2024-09-14T07:09:52.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543043+0000", 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2109999999999997e-06 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3854 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543614+0000", 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0022627440000000001, 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045075849999999997, 2024-09-14T07:09:52.860 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3854 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543614+0000", 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543614+0000", 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543614+0000", 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3700000000000001e-07 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543615+0000", 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4699999999999999e-06 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543619+0000", 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8039999999999999e-06 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543622+0000", 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8279999999999999e-06 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.863 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3898 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545801+0000", 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.5711999999999996e-05, 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023445620000000001, 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.864 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3898 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545801+0000", 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545801+0000", 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545801+0000", 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.34e-07 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545803+0000", 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0540000000000002e-06 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545815+0000", 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2119999999999999e-05 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545821+0000", 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5790000000000003e-06 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3791 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542295+0000", 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0035814319999999998, 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058773540000000004, 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3791 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542295+0000", 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542295+0000", 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542295+0000", 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3799999999999998e-07 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542298+0000", 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2010000000000002e-06 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542303+0000", 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8850000000000004e-06 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542308+0000", 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4839999999999998e-06 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3823 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543048+0000", 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0028279780000000001, 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051482330000000003, 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3823 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543048+0000", 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543048+0000", 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543049+0000", 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4999999999999998e-07 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543050+0000", 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.508e-06 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543054+0000", 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6239999999999999e-06 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543057+0000", 2024-09-14T07:09:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7489999999999999e-06 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3855 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543626+0000", 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002250443, 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00460743, 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3855 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543626+0000", 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543626+0000", 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543626+0000", 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8200000000000001e-07 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543628+0000", 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.209e-06 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543631+0000", 2024-09-14T07:09:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.196e-06 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543633+0000", 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6189999999999998e-06 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3899 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545828+0000", 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.8028000000000002e-05, 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024304370000000001, 2024-09-14T07:09:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3899 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545828+0000", 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545828+0000", 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545829+0000", 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1099999999999996e-07 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545831+0000", 2024-09-14T07:09:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.238e-06 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545844+0000", 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2699e-05 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545849+0000", 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3040000000000004e-06 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3792 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542315+0000", 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003561814, 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0059720579999999997, 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3792 2024-09-14T07:09:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542315+0000", 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542315+0000", 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542315+0000", 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.32e-07 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542317+0000", 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.052e-06 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542323+0000", 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0739999999999998e-06 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542328+0000", 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2429999999999996e-06 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3824 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543061+0000", 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0028157590000000001, 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052503890000000003, 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3824 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543061+0000", 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543061+0000", 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543061+0000", 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6899999999999999e-07 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543062+0000", 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1540000000000001e-06 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543065+0000", 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1650000000000002e-06 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543068+0000", 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6539999999999999e-06 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3856 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543637+0000", 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0022395140000000002, 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047038430000000001, 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3856 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543637+0000", 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543637+0000", 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543637+0000", 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543638+0000", 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.271e-06 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543641+0000", 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0900000000000001e-06 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543644+0000", 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6929999999999999e-06 2024-09-14T07:09:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3900 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545863+0000", 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.3777999999999999e-05, 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025037010000000001, 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3900 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545863+0000", 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545863+0000", 2024-09-14T07:09:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545863+0000", 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2e-07 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545866+0000", 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.48e-06 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545873+0000", 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6059999999999997e-06 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545878+0000", 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6589999999999998e-06 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3793 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542335+0000", 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0035409819999999998, 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0060575239999999999, 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3793 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542335+0000", 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542335+0000", 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542336+0000", 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3500000000000002e-07 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542338+0000", 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.996e-06 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542344+0000", 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7939999999999999e-06 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542349+0000", 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5319999999999997e-06 2024-09-14T07:09:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3825 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543071+0000", 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0028050039999999998, 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0053482110000000003, 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3825 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543071+0000", 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543071+0000", 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543072+0000", 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543073+0000", 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.291e-06 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543076+0000", 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.118e-06 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543079+0000", 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3790000000000002e-06 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3857 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543648+0000", 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0022288360000000001, 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047966409999999999, 2024-09-14T07:09:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3857 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543648+0000", 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543648+0000", 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543648+0000", 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6199999999999999e-07 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543653+0000", 2024-09-14T07:09:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1100000000000002e-06 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543658+0000", 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6959999999999996e-06 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543660+0000", 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7389999999999999e-06 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3901 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545885+0000", 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999909, 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002586189, 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3901 2024-09-14T07:09:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545885+0000", 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545885+0000", 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545886+0000", 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4799999999999999e-07 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545888+0000", 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545894+0000", 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.178e-06 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545899+0000", 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7160000000000002e-06 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3794 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542356+0000", 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003520357, 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0061433770000000002, 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3794 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542356+0000", 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542356+0000", 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542356+0000", 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1800000000000001e-07 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542359+0000", 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0820000000000001e-06 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542365+0000", 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9299999999999997e-06 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542371+0000", 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4280000000000004e-06 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3826 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543083+0000", 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002793687, 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0054438519999999999, 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3826 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543083+0000", 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543083+0000", 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543083+0000", 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7399999999999999e-07 2024-09-14T07:09:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543084+0000", 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4449999999999999e-06 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543087+0000", 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0819999999999999e-06 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543090+0000", 2024-09-14T07:09:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5639999999999999e-06 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3858 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543664+0000", 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0022120680000000002, 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0048877800000000004, 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3858 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543664+0000", 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543664+0000", 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543665+0000", 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.84e-07 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543666+0000", 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543670+0000", 2024-09-14T07:09:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.625e-06 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543672+0000", 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6570000000000001e-06 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3902 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545916+0000", 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999599, 2024-09-14T07:09:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026639049999999998, 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3902 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545916+0000", 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545916+0000", 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545917+0000", 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6100000000000001e-07 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545919+0000", 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545927+0000", 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6109999999999996e-06 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545932+0000", 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0599999999999998e-06 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3795 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542377+0000", 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0034989779999999998, 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0062288450000000002, 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3795 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542377+0000", 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542377+0000", 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542378+0000", 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2599999999999998e-07 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542380+0000", 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0329999999999998e-06 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542386+0000", 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8889999999999999e-06 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542391+0000", 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8369999999999996e-06 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3827 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543106+0000", 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002770485, 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0055269150000000003, 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3827 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543106+0000", 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543106+0000", 2024-09-14T07:09:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543107+0000", 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.16e-07 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543109+0000", 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543115+0000", 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0580000000000003e-06 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543120+0000", 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0679999999999996e-06 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3859 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543676+0000", 2024-09-14T07:09:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0022008079999999998, 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0049818600000000003, 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3859 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543676+0000", 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543676+0000", 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543676+0000", 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543677+0000", 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3170000000000001e-06 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543680+0000", 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0139999999999999e-06 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543683+0000", 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6369999999999999e-06 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3903 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545961+0000", 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999156, 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027233980000000001, 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3903 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545961+0000", 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545961+0000", 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545961+0000", 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9299999999999999e-07 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545963+0000", 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.736e-06 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545967+0000", 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2740000000000001e-06 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545970+0000", 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1360000000000001e-06 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3796 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542397+0000", 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0034792909999999998, 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0063141450000000002, 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3796 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542397+0000", 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542397+0000", 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542398+0000", 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3700000000000001e-07 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542400+0000", 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0370000000000001e-06 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542405+0000", 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8130000000000001e-06 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542410+0000", 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3828 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543127+0000", 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0027493439999999999, 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056116020000000003, 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3828 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543127+0000", 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543127+0000", 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543128+0000", 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-07 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543130+0000", 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.996e-06 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543135+0000", 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8170000000000004e-06 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543141+0000", 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6119999999999998e-06 2024-09-14T07:09:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3860 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543686+0000", 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0021899670000000001, 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050780970000000002, 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3860 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543686+0000", 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543686+0000", 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543687+0000", 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.48e-07 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543688+0000", 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.31e-06 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543692+0000", 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2010000000000001e-06 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543695+0000", 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6259999999999999e-06 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3904 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545975+0000", 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999013, 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028161869999999999, 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3904 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545975+0000", 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545975+0000", 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545976+0000", 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2100000000000002e-07 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545977+0000", 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.291e-06 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545980+0000", 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.23e-06 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545983+0000", 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7209999999999999e-06 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3797 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542417+0000", 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0034596140000000002, 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064006330000000002, 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3797 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542417+0000", 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542417+0000", 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542417+0000", 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3599999999999999e-07 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542432+0000", 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4357e-05 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542439+0000", 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2869999999999999e-06 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542444+0000", 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9590000000000003e-06 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3829 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543147+0000", 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0027290270000000002, 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056953489999999997, 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3829 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543147+0000", 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543147+0000", 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543148+0000", 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543150+0000", 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9879999999999999e-06 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543156+0000", 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2060000000000004e-06 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543161+0000", 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5589999999999999e-06 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3861 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543698+0000", 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0021781550000000002, 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051707339999999997, 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.955 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3861 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543698+0000", 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543698+0000", 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543698+0000", 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5199999999999998e-07 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543700+0000", 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.167e-06 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543703+0000", 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1080000000000001e-06 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543706+0000", 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.585e-06 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3905 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.958 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545986+0000", 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998899, 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029077529999999999, 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3905 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.959 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545986+0000", 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545986+0000", 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545987+0000", 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.53e-07 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545988+0000", 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2100000000000001e-06 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545991+0000", 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1240000000000001e-06 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545994+0000", 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6170000000000001e-06 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3798 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542451+0000", 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003424954, 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064706010000000003, 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.962 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3798 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542451+0000", 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542451+0000", 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542452+0000", 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7700000000000005e-07 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542454+0000", 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.148e-06 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542460+0000", 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8470000000000001e-06 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542465+0000", 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6650000000000002e-06 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3830 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543167+0000", 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0027094430000000002, 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057807639999999999, 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3830 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543167+0000", 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543167+0000", 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543167+0000", 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4000000000000002e-07 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543169+0000", 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9439999999999999e-06 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543175+0000", 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0379999999999996e-06 2024-09-14T07:09:52.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543180+0000", 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5469999999999998e-06 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3862 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543727+0000", 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0021492080000000001, 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052462409999999996, 2024-09-14T07:09:52.969 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3862 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543727+0000", 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543727+0000", 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543728+0000", 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2500000000000005e-07 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543730+0000", 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.176e-06 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543736+0000", 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8850000000000004e-06 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543740+0000", 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.622e-06 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3906 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545997+0000", 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998794, 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003000876, 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3906 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545997+0000", 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545997+0000", 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545997+0000", 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:09:52.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546003+0000", 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7769999999999999e-06 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546007+0000", 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4970000000000001e-06 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546009+0000", 2024-09-14T07:09:52.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3799 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542471+0000", 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003405417, 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065560189999999997, 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3799 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542471+0000", 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542471+0000", 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542471+0000", 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542473+0000", 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.029e-06 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542479+0000", 2024-09-14T07:09:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5940000000000001e-06 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542484+0000", 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7110000000000003e-06 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3831 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543186+0000", 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026904979999999999, 2024-09-14T07:09:52.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058653480000000003, 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3831 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543186+0000", 2024-09-14T07:09:52.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543186+0000", 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543186+0000", 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.32e-07 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543188+0000", 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.818e-06 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543194+0000", 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6690000000000003e-06 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543198+0000", 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6700000000000002e-06 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3863 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543748+0000", 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002128451, 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0053291299999999996, 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3863 2024-09-14T07:09:52.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543748+0000", 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543748+0000", 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543748+0000", 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543750+0000", 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1639999999999999e-06 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543756+0000", 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6949999999999997e-06 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543766+0000", 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3510000000000008e-06 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3907 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.546013+0000", 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998627, 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030883109999999998, 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.986 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3907 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546013+0000", 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546013+0000", 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546014+0000", 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7799999999999997e-07 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546015+0000", 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.248e-06 2024-09-14T07:09:52.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546018+0000", 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0479999999999999e-06 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546021+0000", 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6970000000000002e-06 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3800 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542490+0000", 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003386329, 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006638934, 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3800 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.990 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542490+0000", 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542490+0000", 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542490+0000", 2024-09-14T07:09:52.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542492+0000", 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.852e-06 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542500+0000", 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.289e-06 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542504+0000", 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5940000000000004e-06 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3832 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543205+0000", 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026717389999999998, 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0059494329999999996, 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.993 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3832 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543205+0000", 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543205+0000", 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543205+0000", 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0400000000000002e-07 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543207+0000", 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543213+0000", 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543218+0000", 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5589999999999999e-06 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3864 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:52.996 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.544977+0000", 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00089892100000000001, 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042021580000000001, 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3864 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.544977+0000", 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.544977+0000", 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.544978+0000", 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9100000000000003e-07 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.544981+0000", 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2940000000000001e-06 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.544993+0000", 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1891e-05 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.544999+0000", 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1930000000000001e-06 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:52.999 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3908 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.546024+0000", 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998522, 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031799290000000002, 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.000 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3908 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546024+0000", 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546024+0000", 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546025+0000", 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.79e-07 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546026+0000", 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.217e-06 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546029+0000", 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:53.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546031+0000", 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.599e-06 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3801 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542511+0000", 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.003365079, 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006718674, 2024-09-14T07:09:53.003 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3801 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542511+0000", 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542511+0000", 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542512+0000", 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7599999999999997e-07 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542514+0000", 2024-09-14T07:09:53.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1409999999999999e-06 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542520+0000", 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6879999999999996e-06 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542524+0000", 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3590000000000001e-06 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3833 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543225+0000", 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026515919999999999, 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0060313850000000002, 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3833 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.007 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543225+0000", 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543225+0000", 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543225+0000", 2024-09-14T07:09:53.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.08e-07 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543232+0000", 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.477e-06 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543246+0000", 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4246999999999999e-05 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543252+0000", 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4690000000000002e-06 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3865 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545008+0000", 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00086793299999999999, 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042718449999999998, 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3865 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545008+0000", 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545008+0000", 2024-09-14T07:09:53.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545009+0000", 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7899999999999999e-07 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545011+0000", 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8190000000000001e-06 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545015+0000", 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2869999999999998e-06 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545020+0000", 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6909999999999996e-06 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3909 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.013 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.546043+0000", 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998336, 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032644760000000001, 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3909 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546043+0000", 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546043+0000", 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546043+0000", 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3799999999999998e-07 2024-09-14T07:09:53.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546045+0000", 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8479999999999999e-06 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546050+0000", 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.164e-06 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546053+0000", 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9670000000000001e-06 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3802 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.542531+0000", 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0033449999999999999, 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0068010900000000001, 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3802 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542531+0000", 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542531+0000", 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542532+0000", 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.77e-07 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542534+0000", 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1469999999999999e-06 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542540+0000", 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.161e-06 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.542545+0000", 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0320000000000003e-06 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.020 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3834 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.543261+0000", 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026156949999999999, 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0060974339999999997, 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3834 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543261+0000", 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543261+0000", 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543261+0000", 2024-09-14T07:09:53.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6400000000000003e-07 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543264+0000", 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543270+0000", 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.543275+0000", 2024-09-14T07:09:53.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3619999999999997e-06 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3866 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.545033+0000", 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00084365199999999997, 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043497450000000003, 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.024 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3866 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545033+0000", 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545033+0000", 2024-09-14T07:09:53.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545033+0000", 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6000000000000003e-07 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545036+0000", 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545042+0000", 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4139999999999999e-06 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.545047+0000", 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0640000000000001e-06 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5620.0:3910 3.1c 3.7e990b7c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e115)", 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:52.546058+0000", 2024-09-14T07:09:53.027 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998183, 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033508280000000001, 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5620", 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3755204665", 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 3910 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546058+0000", 2024-09-14T07:09:53.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546058+0000", 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546059+0000", 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6000000000000003e-07 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546061+0000", 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.911e-06 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546067+0000", 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8540000000000002e-06 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:52.546072+0000", 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.972e-06 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.030 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.031 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:09:53.031 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 127 2024-09-14T07:09:53.031 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:53.032 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:52.559+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:53.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:53.033 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:53.031+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:53.132 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:53.131+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:53.233 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:53.231+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:53.236 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:53.236 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:53.236 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:53.236 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:53.237 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:53.254 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:53.339 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:53.339+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:53.435 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:53.435+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:53.443 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:53.443 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:53.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:53.443 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:53.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.443 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:53.443 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.611424692, 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.61065034400001, 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.449 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:09:53.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 91.599467345999997, 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:09:53.454 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:53.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:09:53.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:09:53.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 91.302539139999993, 2024-09-14T07:09:53.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:09:53.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:09:53.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.067635316000001, 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:09:53.466 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:09:53.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:09:53.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.067073524999998, 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:09:53.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.066578958000001, 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:09:53.475 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:09:53.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:09:53.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.065961539, 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.480 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:09:53.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.065533279999997, 2024-09-14T07:09:53.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:09:53.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.065137415999999, 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:09:53.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:09:53.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.06454574, 2024-09-14T07:09:53.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:09:53.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.061216625999997, 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.061175575, 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.502 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:09:53.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:09:53.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.061123315000003, 2024-09-14T07:09:53.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:09:53.510 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.061084428000001, 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.511 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:09:53.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.061025084999997, 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:09:53.515 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:53.519 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.060970996999998, 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:09:53.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:09:53.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.523 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.060931644, 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.527 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.060893864000001, 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:09:53.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.060857161000001, 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:53.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:09:53.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:09:53.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:53.536 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:53.537 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:53.539 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T07:09:53.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T07:09:53.539 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:53.535+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:53.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:53.639 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:53.635+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:53.713 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:53.713 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:53.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:53.740 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:53.739+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:53.841 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:53.839+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:53.848 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:53.849 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:53.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:53.942 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:53.939+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 96.701719572000002, 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 96.672211941, 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 96.663564582000006, 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.997472242000001, 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.830743016, 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.897742007999994, 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.142298836000002, 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:09:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.142280913999997, 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:09:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.142268145000003, 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:09:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.142257968999999, 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:09:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:09:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.142247773999998, 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:09:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.142237754, 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.08873664, 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9284291280000003, 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9283975890000002, 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:09:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:09:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9283662770000003, 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9283263000000002, 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:09:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:09:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9193154809999999, 2024-09-14T07:09:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:09:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3092 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071264+0000", 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9191961969999998, 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560395999999999, 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3092 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071264+0000", 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071264+0000", 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071265+0000", 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.08e-06 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071271+0000", 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8629999999999996e-06 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071290+0000", 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.836e-05 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071305+0000", 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4932999999999999e-05 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111796+0000", 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040491025999999999 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111825+0000", 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9133999999999999e-05 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9187059170000005, 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:09:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:09:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:54.086 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:54.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:54.087 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:54.087+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:54.188 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:54.187+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:54.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:54.265 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:54.265 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:54.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:54.327 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:54.327+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:54.367 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:54.367 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:54.384 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:54.389 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:54.387+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:54.400 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:54.400 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:54.401 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:54.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:54.421 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:54.444 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:54.462 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:54.463 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:54.490 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:54.487+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:54.499 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T07:09:54.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:09:54.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T07:09:54.543 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:54.543 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:54.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:54.607 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:54.607+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:54.693 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:54.691+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:54.759 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:54.760 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:54.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:54.815 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:54.815+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:54.895 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:54.891+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:54.929 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:54.930 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:54.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:55.035 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:55.035+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:55.096 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:55.095+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:55.113 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:55.113 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:55.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:55.196 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:55.195+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:55.262 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:55.262 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:55.279 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:55.298 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:55.295+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:55.320 INFO:teuthology.orchestra.run.smithi145.stdout:4177 read 0x151ce00 thru 0x1528b26 (0xbd27 bytes) 2024-09-14T07:09:55.398 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:55.395+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:55.445 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:55.445 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:55.459 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:55.500 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:55.499+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:55.504 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:09:55.509 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T07:09:55.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T07:09:55.627 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:55.627+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:55.629 INFO:teuthology.orchestra.run.smithi145.stdout:4179 write 0xa825000 thru 0xa8253af (0x3b0 bytes) 2024-09-14T07:09:55.629 INFO:teuthology.orchestra.run.smithi145.stdout:4180 write 0x4301a00 thru 0x4303d6c (0x236d bytes) 2024-09-14T07:09:55.631 INFO:teuthology.orchestra.run.smithi145.stdout:4181 punch from 0x14d4800 to 0x14d8200, (0x3a00 bytes) 2024-09-14T07:09:55.636 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:55.636 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:55.652 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:55.701 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:55.699+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:55.717 INFO:teuthology.orchestra.run.smithi145.stdout:4185 trunc from 0xa825400 to 0x215800 2024-09-14T07:09:55.787 INFO:teuthology.orchestra.run.smithi145.stdout:4186 write 0x9cd400 thru 0x9d0310 (0x2f11 bytes) 2024-09-14T07:09:55.789 INFO:teuthology.orchestra.run.smithi145.stdout:4187 punch from 0x245e00 to 0x254c00, (0xee00 bytes) 2024-09-14T07:09:55.789 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:55.790 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:55.802 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:55.799+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:55.806 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:55.811 INFO:teuthology.orchestra.run.smithi145.stdout:4188 write 0x5247c00 thru 0x52508a0 (0x8ca1 bytes) 2024-09-14T07:09:55.812 INFO:teuthology.orchestra.run.smithi145.stdout:4189 write 0x25ee200 thru 0x25f3627 (0x5428 bytes) 2024-09-14T07:09:55.812 INFO:teuthology.orchestra.run.smithi145.stdout:4190 read 0x23bb200 thru 0x23c5423 (0xa224 bytes) 2024-09-14T07:09:55.812 INFO:teuthology.orchestra.run.smithi145.stdout:4191 read 0x4ccf800 thru 0x4cd7c5c (0x845d bytes) 2024-09-14T07:09:55.812 INFO:teuthology.orchestra.run.smithi145.stdout:4195 read 0x239d800 thru 0x23a3655 (0x5e56 bytes) 2024-09-14T07:09:55.813 INFO:teuthology.orchestra.run.smithi145.stdout:4196 clone 30 order 21 su 262144 sc 4 2024-09-14T07:09:55.903 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:55.903+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:55.924 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:55.924 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:55.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:56.004 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:56.003+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:56.059 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:56.060 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:56.073 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:56.116 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:56.115+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:56.206 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:56.203+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:56.239 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:56.239 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:56.254 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:56.344 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:56.343+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:56.407 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:56.407+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:56.417 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:56.417 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:56.431 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:56.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:09:56.520 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:56.519+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:56.521 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T07:09:56.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T07:09:56.599 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:56.600 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:56.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:09:56.631 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:56.631+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:56.641 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:56.641 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:56.677 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:56.677 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:56.687 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone29 from 0x5250a00 (overlap 0x215800) to 0x10ac00 2024-09-14T07:09:56.710 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:56.707+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:56.711 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:56.750 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:56.785 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:56.786 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:56.802 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:56.803 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:56.811 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:56.807+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:56.822 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:09:56.822 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:56.823 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:56.912 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:56.911+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:56.947 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:56.947 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:56.947 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:56.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:09:57.012 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:57.011+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:57.113 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:57.111+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:57.134 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 99.843139477999998, 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:09:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 99.813631846999996, 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:09:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 99.804984488000002, 2024-09-14T07:09:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:09:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 95.138892147999996, 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:09:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 94.972162921999995, 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.039161914000005, 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:09:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:09:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:09:57.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.283718741999998, 2024-09-14T07:09:57.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:57.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:57.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.168 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.28370082, 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.169 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:09:57.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:09:57.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:09:57.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.283688050999999, 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:57.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.283677875000002, 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:09:57.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.283667680000001, 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:09:57.181 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:09:57.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:09:57.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.283657660000003, 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:09:57.185 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:09:57.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:09:57.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.230156546000003, 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:09:57.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0698490340000006, 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:09:57.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:09:57.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:09:57.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0698174950000006, 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:09:57.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0697861829999997, 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:09:57.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:09:57.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0697462059999996, 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:09:57.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:09:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0607353869999994, 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:09:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3092 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071264+0000", 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0606161029999992, 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560395999999999, 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3092 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071264+0000", 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071264+0000", 2024-09-14T07:09:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071265+0000", 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.08e-06 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071271+0000", 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8629999999999996e-06 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071290+0000", 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.836e-05 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071305+0000", 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4932999999999999e-05 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111796+0000", 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040491025999999999 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111825+0000", 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9133999999999999e-05 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0601258229999999, 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:09:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:09:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:09:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.223 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.223 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.223 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:57.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:09:57.225 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:57.223+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:57.325 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:57.323+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:57.400 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:57.401 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:57.401 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:57.401 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:57.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:09:57.425 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:57.423+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:57.523 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T07:09:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T07:09:57.526 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:57.523+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:57.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:09:57.546 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T07:09:57.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:57.590 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:57.590 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:57.590 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:57.590 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:57.591 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:57.594 INFO:teuthology.orchestra.run.smithi145.stdout:4197 punch from 0x63f600 to 0x64ee00, (0xf800 bytes) 2024-09-14T07:09:57.610 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:09:57.651 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:57.651+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:57.705 INFO:teuthology.orchestra.run.smithi145.stdout:4199 write 0xad01a00 thru 0xad0d7b2 (0xbdb3 bytes) 2024-09-14T07:09:57.707 INFO:teuthology.orchestra.run.smithi145.stdout:4203 read 0x254c000 thru 0x2554c88 (0x8c89 bytes) 2024-09-14T07:09:57.708 INFO:teuthology.orchestra.run.smithi145.stdout:4204 read 0x1ca1400 thru 0x1ca8266 (0x6e67 bytes) 2024-09-14T07:09:57.728 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:57.727+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:57.761 INFO:teuthology.orchestra.run.smithi145.stdout:4208 write 0xdd21000 thru 0xdd23b5d (0x2b5e bytes) 2024-09-14T07:09:57.763 INFO:teuthology.orchestra.run.smithi145.stdout:4209 punch from 0x929a800 to 0x92a7c00, (0xd400 bytes) 2024-09-14T07:09:57.764 INFO:teuthology.orchestra.run.smithi145.stdout:4210 punch from 0xc7d1800 to 0xc7d2a00, (0x1200 bytes) 2024-09-14T07:09:57.764 INFO:teuthology.orchestra.run.smithi145.stdout:4211 punch from 0xba08800 to 0xba17c00, (0xf400 bytes) 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 96.103271559000007, 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:09:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.712295722, 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.712265924, 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:09:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.712233648999998, 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:09:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.712175545999997, 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:09:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.712134644000002, 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:09:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.712072835999997, 2024-09-14T07:09:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.71203852, 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:09:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:09:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.712019849000001, 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711995813000001, 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711975778999999, 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:57.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711960943999998, 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:09:57.831 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:57.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:09:57.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:57.835 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711946318000003, 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:57.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:09:57.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711931817, 2024-09-14T07:09:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:09:57.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:09:57.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711914061999998, 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:09:57.843 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711899168000002, 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:09:57.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:57.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711885057000003, 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:57.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:09:57.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711863127999997, 2024-09-14T07:09:57.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:57.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:57.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.711841817, 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:57.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:09:57.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.424490916000003, 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:09:57.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:09:57.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:57.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:57.869 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:57.872 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4216 trunc from 0xdd23c00 to 0x3589000 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4221 write 0xce3f400 thru 0xce4e143 (0xed44 bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4223 punch from 0x88b5a00 to 0x88c2e00, (0xd400 bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4225 punch from 0x3996400 to 0x39a2a00, (0xc600 bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4226 read 0x16b1600 thru 0x16b223b (0xc3c bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4228 write 0x93f9c00 thru 0x94063fe (0xc7ff bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4231 read 0x7fbce00 thru 0x7fc623b (0x943c bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4233 read 0x19e5400 thru 0x19eb728 (0x6329 bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4234 write 0x9e97a00 thru 0x9e9ee35 (0x7436 bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4237 punch from 0x4e6d600 to 0x4e7ac00, (0xd600 bytes) 2024-09-14T07:09:57.873 INFO:teuthology.orchestra.run.smithi145.stdout:4238 read 0x9d8f200 thru 0x9d9b6a9 (0xc4aa bytes) 2024-09-14T07:09:57.874 INFO:teuthology.orchestra.run.smithi145.stdout:4240 write 0x7904a00 thru 0x790d6d2 (0x8cd3 bytes) 2024-09-14T07:09:57.874 INFO:teuthology.orchestra.run.smithi145.stdout:4241 read 0x34e3800 thru 0x34f2243 (0xea44 bytes) 2024-09-14T07:09:57.874 INFO:teuthology.orchestra.run.smithi145.stdout:4243 clone 31 order 23 su 2097152 sc 12 2024-09-14T07:09:57.874 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:57.867+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:57.939 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:57.962 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-14T07:09:57.521418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1628342422,"num_objects":875,"num_object_clones":143,"num_object_copies":1750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":875,"num_whiteouts":234,"num_read":249716,"num_read_kb":766485,"num_write":11669,"num_write_kb":259468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1004,"num_bytes_recovered":128743440,"num_keys_recovered":288,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9578,"ondisk_log_size":9578,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":219732,"kb_used_data":40900,"kb_used_omap":9,"kb_used_meta":178678,"kb_avail":566011308,"statfs":{"total":579820584960,"available":579595579392,"internally_reserved":0,"allocated":41881600,"data_stored":312713010,"data_compressed":13079300,"data_compressed_allocated":27619328,"data_compressed_original":306106880,"omap_allocated":9568,"internal_metadata":182966944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,129,114,129],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25649314,"num_objects":30,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":18238,"num_read_kb":26026,"num_write":252,"num_write_kb":1078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025037"},"pg_stats":[{"pgid":"3.1f","version":"113'148","reported_seq":5065,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.018441+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:09:55.018441+0000","last_peered":"2024-09-14T07:09:55.018441+0000","last_clean":"2024-09-14T07:09:55.018441+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:09:55.018441+0000","last_undegraded":"2024-09-14T07:09:55.018441+0000","last_fullsized":"2024-09-14T07:09:55.018441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:37.693843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":69978128,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":4454,"num_read_kb":43252,"num_write":232,"num_write_kb":10177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"113'450","reported_seq":8421,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.225739+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:55.225739+0000","last_peered":"2024-09-14T07:09:55.225739+0000","last_clean":"2024-09-14T07:09:55.225739+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:55.225739+0000","last_undegraded":"2024-09-14T07:09:55.225739+0000","last_fullsized":"2024-09-14T07:09:55.225739+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:08:04.096796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":66759046,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":9323,"num_read_kb":40106,"num_write":734,"num_write_kb":13123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"114'600","reported_seq":8898,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522670+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:54.522670+0000","last_peered":"2024-09-14T07:09:54.522670+0000","last_clean":"2024-09-14T07:09:54.522670+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:54.522670+0000","last_undegraded":"2024-09-14T07:09:54.522670+0000","last_fullsized":"2024-09-14T07:09:54.522670+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:06.985688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":10296420,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":11640,"num_read_kb":20380,"num_write":411,"num_write_kb":2962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"114'176","reported_seq":13801,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.243958+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:55.243958+0000","last_peered":"2024-09-14T07:09:55.243958+0000","last_clean":"2024-09-14T07:09:55.243958+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:55.243958+0000","last_undegraded":"2024-09-14T07:09:55.243958+0000","last_fullsized":"2024-09-14T07:09:55.243958+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:12:37.762139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":37774900,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":13469,"num_read_kb":20176,"num_write":191,"num_write_kb":4683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"109'377","reported_seq":7702,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.018654+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:09:55.018654+0000","last_peered":"2024-09-14T07:09:55.018654+0000","last_clean":"2024-09-14T07:09:55.018654+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:09:55.018654+0000","last_undegraded":"2024-09-14T07:09:55.018654+0000","last_fullsized":"2024-09-14T07:09:55.018654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:22.272156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":102915259,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":8067,"num_read_kb":21163,"num_write":512,"num_write_kb":10477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"114'217","reported_seq":27104,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:57.082789+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:57.082789+0000","last_peered":"2024-09-14T07:09:57.082789+0000","last_clean":"2024-09-14T07:09:57.082789+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:57.082789+0000","last_undegraded":"2024-09-14T07:09:57.082789+0000","last_fullsized":"2024-09-14T07:09:57.082789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:16:36.796265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":81449645,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":27691,"num_read_kb":46889,"num_write":260,"num_write_kb":10858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"114'179","reported_seq":5340,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499719+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:54.499719+0000","last_peered":"2024-09-14T07:09:54.499719+0000","last_clean":"2024-09-14T07:09:54.499719+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:54.499719+0000","last_undegraded":"2024-09-14T07:09:54.499719+0000","last_fullsized":"2024-09-14T07:09:54.499719+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:26:24.800199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":30128251,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":5081,"num_read_kb":12880,"num_write":198,"num_write_kb":5982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"114'169","reported_seq":2603,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.223771+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:09:55.223771+0000","last_peered":"2024-09-14T07:09:55.223771+0000","last_clean":"2024-09-14T07:09:55.223771+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:09:55.223771+0000","last_undegraded":"2024-09-14T07:09:55.223771+0000","last_fullsized":"2024-09-14T07:09:55.223771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:47:37.142427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":76865736,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":2225,"num_read_kb":6268,"num_write":287,"num_write_kb":13843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"109'116","reported_seq":1926,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522837+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:09:54.522837+0000","last_peered":"2024-09-14T07:09:54.522837+0000","last_clean":"2024-09-14T07:09:54.522837+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:09:54.522837+0000","last_undegraded":"2024-09-14T07:09:54.522837+0000","last_fullsized":"2024-09-14T07:09:54.522837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:18:52.224542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":37352486,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1465,"num_read_kb":9238,"num_write":132,"num_write_kb":2767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"109'252","reported_seq":7108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522737+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:54.522737+0000","last_peered":"2024-09-14T07:09:54.522737+0000","last_clean":"2024-09-14T07:09:54.522737+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:54.522737+0000","last_undegraded":"2024-09-14T07:09:54.522737+0000","last_fullsized":"2024-09-14T07:09:54.522737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:09.242953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":48148006,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":7629,"num_read_kb":32754,"num_write":224,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"112'255","reported_seq":4435,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.228808+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:09:55.228808+0000","last_peered":"2024-09-14T07:09:55.228808+0000","last_clean":"2024-09-14T07:09:55.228808+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:09:55.228808+0000","last_undegraded":"2024-09-14T07:09:55.228808+0000","last_fullsized":"2024-09-14T07:09:55.228808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:58.984154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":67224625,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":5248,"num_read_kb":38739,"num_write":320,"num_write_kb":13909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"114'509","reported_seq":8721,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500997+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:54.500997+0000","last_peered":"2024-09-14T07:09:54.500997+0000","last_clean":"2024-09-14T07:09:54.500997+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:54.500997+0000","last_undegraded":"2024-09-14T07:09:54.500997+0000","last_fullsized":"2024-09-14T07:09:54.500997+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:20:28.164748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":59498099,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":9948,"num_read_kb":31183,"num_write":715,"num_write_kb":13153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":229,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501334+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:54.501334+0000","last_peered":"2024-09-14T07:09:54.501334+0000","last_clean":"2024-09-14T07:09:54.501334+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:54.501334+0000","last_undegraded":"2024-09-14T07:09:54.501334+0000","last_fullsized":"2024-09-14T07:09:54.501334+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'428","reported_seq":9319,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.018233+0000","last_change":"2024-09-14T07:09:14.793462+0000","last_active":"2024-09-14T07:09:55.018233+0000","last_peered":"2024-09-14T07:09:55.018233+0000","last_clean":"2024-09-14T07:09:55.018233+0000","last_became_active":"2024-09-14T07:09:14.787124+0000","last_became_peered":"2024-09-14T07:09:14.787124+0000","last_unstale":"2024-09-14T07:09:55.018233+0000","last_undegraded":"2024-09-14T07:09:55.018233+0000","last_fullsized":"2024-09-14T07:09:55.018233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:54:53.102179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":21480210,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":6,"num_read":11359,"num_read_kb":21400,"num_write":511,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1461720,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499856+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:54.499856+0000","last_peered":"2024-09-14T07:09:54.499856+0000","last_clean":"2024-09-14T07:09:54.499856+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:54.499856+0000","last_undegraded":"2024-09-14T07:09:54.499856+0000","last_fullsized":"2024-09-14T07:09:54.499856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"113'205","reported_seq":4289,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.540585+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:54.540585+0000","last_peered":"2024-09-14T07:09:54.540585+0000","last_clean":"2024-09-14T07:09:54.540585+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:54.540585+0000","last_undegraded":"2024-09-14T07:09:54.540585+0000","last_fullsized":"2024-09-14T07:09:54.540585+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:36:39.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":33995806,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":4605,"num_read_kb":13735,"num_write":248,"num_write_kb":2656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":229,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501182+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:09:54.501182+0000","last_peered":"2024-09-14T07:09:54.501182+0000","last_clean":"2024-09-14T07:09:54.501182+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:09:54.501182+0000","last_undegraded":"2024-09-14T07:09:54.501182+0000","last_fullsized":"2024-09-14T07:09:54.501182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"113'460","reported_seq":4675,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.231694+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:55.231694+0000","last_peered":"2024-09-14T07:09:55.231694+0000","last_clean":"2024-09-14T07:09:55.231694+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:55.231694+0000","last_undegraded":"2024-09-14T07:09:55.231694+0000","last_fullsized":"2024-09-14T07:09:55.231694+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:16:38.943523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":4813879,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":6364,"num_read_kb":9719,"num_write":667,"num_write_kb":2103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":229,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500020+0000","last_change":"2024-09-14T07:09:11.349379+0000","last_active":"2024-09-14T07:09:54.500020+0000","last_peered":"2024-09-14T07:09:54.500020+0000","last_clean":"2024-09-14T07:09:54.500020+0000","last_became_active":"2024-09-14T07:09:11.348941+0000","last_became_peered":"2024-09-14T07:09:11.348941+0000","last_unstale":"2024-09-14T07:09:54.500020+0000","last_undegraded":"2024-09-14T07:09:54.500020+0000","last_fullsized":"2024-09-14T07:09:54.500020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:39.931750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"114'529","reported_seq":2629,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522789+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:54.522789+0000","last_peered":"2024-09-14T07:09:54.522789+0000","last_clean":"2024-09-14T07:09:54.522789+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:54.522789+0000","last_undegraded":"2024-09-14T07:09:54.522789+0000","last_fullsized":"2024-09-14T07:09:54.522789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:10:04.835880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":13788461,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":3,"num_read":4187,"num_read_kb":16563,"num_write":919,"num_write_kb":4981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"113'116","reported_seq":6299,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:56.523828+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:09:56.523828+0000","last_peered":"2024-09-14T07:09:56.523828+0000","last_clean":"2024-09-14T07:09:56.523828+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:09:56.523828+0000","last_undegraded":"2024-09-14T07:09:56.523828+0000","last_fullsized":"2024-09-14T07:09:56.523828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:09:24.740371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":63791616,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":5613,"num_read_kb":24884,"num_write":197,"num_write_kb":12074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":219,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522836+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:54.522836+0000","last_peered":"2024-09-14T07:09:54.522836+0000","last_clean":"2024-09-14T07:09:54.522836+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:54.522836+0000","last_undegraded":"2024-09-14T07:09:54.522836+0000","last_fullsized":"2024-09-14T07:09:54.522836+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"113'272","reported_seq":3521,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499457+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:54.499457+0000","last_peered":"2024-09-14T07:09:54.499457+0000","last_clean":"2024-09-14T07:09:54.499457+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:54.499457+0000","last_undegraded":"2024-09-14T07:09:54.499457+0000","last_fullsized":"2024-09-14T07:09:54.499457+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:11.014271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":27308868,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":6,"num_read":5320,"num_read_kb":11387,"num_write":196,"num_write_kb":2794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":193,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.540321+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:54.540321+0000","last_peered":"2024-09-14T07:09:54.540321+0000","last_clean":"2024-09-14T07:09:54.540321+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:54.540321+0000","last_undegraded":"2024-09-14T07:09:54.540321+0000","last_fullsized":"2024-09-14T07:09:54.540321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'191","reported_seq":9496,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:56.523896+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:09:56.523896+0000","last_peered":"2024-09-14T07:09:56.523896+0000","last_clean":"2024-09-14T07:09:56.523896+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:09:56.523896+0000","last_undegraded":"2024-09-14T07:09:56.523896+0000","last_fullsized":"2024-09-14T07:09:56.523896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:12.502326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":19655209,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":9101,"num_read_kb":9780,"num_write":172,"num_write_kb":1964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":217,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500844+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:54.500844+0000","last_peered":"2024-09-14T07:09:54.500844+0000","last_clean":"2024-09-14T07:09:54.500844+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:54.500844+0000","last_undegraded":"2024-09-14T07:09:54.500844+0000","last_fullsized":"2024-09-14T07:09:54.500844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'189","reported_seq":5044,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499566+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:54.499566+0000","last_peered":"2024-09-14T07:09:54.499566+0000","last_clean":"2024-09-14T07:09:54.499566+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:54.499566+0000","last_undegraded":"2024-09-14T07:09:54.499566+0000","last_fullsized":"2024-09-14T07:09:54.499566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:46:40.422390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":50891808,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":4675,"num_read_kb":7307,"num_write":174,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":238,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522679+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:54.522679+0000","last_peered":"2024-09-14T07:09:54.522679+0000","last_clean":"2024-09-14T07:09:54.522679+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:54.522679+0000","last_undegraded":"2024-09-14T07:09:54.522679+0000","last_fullsized":"2024-09-14T07:09:54.522679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":332,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500973+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:09:54.500973+0000","last_peered":"2024-09-14T07:09:54.500973+0000","last_clean":"2024-09-14T07:09:54.500973+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:09:54.500973+0000","last_undegraded":"2024-09-14T07:09:54.500973+0000","last_fullsized":"2024-09-14T07:09:54.500973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"114'235","reported_seq":7983,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.227010+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:55.227010+0000","last_peered":"2024-09-14T07:09:55.227010+0000","last_clean":"2024-09-14T07:09:55.227010+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:55.227010+0000","last_undegraded":"2024-09-14T07:09:55.227010+0000","last_fullsized":"2024-09-14T07:09:55.227010+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:33:42.242093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":144767060,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":5,"num_read":7483,"num_read_kb":59764,"num_write":383,"num_write_kb":25537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'430","reported_seq":5082,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:56.524089+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:56.524089+0000","last_peered":"2024-09-14T07:09:56.524089+0000","last_clean":"2024-09-14T07:09:56.524089+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:56.524089+0000","last_undegraded":"2024-09-14T07:09:56.524089+0000","last_fullsized":"2024-09-14T07:09:56.524089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:37:26.541095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":82193820,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":6168,"num_read_kb":39944,"num_write":494,"num_write_kb":16026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"114'196","reported_seq":5860,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501183+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:54.501183+0000","last_peered":"2024-09-14T07:09:54.501183+0000","last_clean":"2024-09-14T07:09:54.501183+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:54.501183+0000","last_undegraded":"2024-09-14T07:09:54.501183+0000","last_fullsized":"2024-09-14T07:09:54.501183+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:02:30.086221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":33249808,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":5522,"num_read_kb":23167,"num_write":202,"num_write_kb":2786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'786","reported_seq":5716,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499896+0000","last_change":"2024-09-14T07:09:14.798144+0000","last_active":"2024-09-14T07:09:54.499896+0000","last_peered":"2024-09-14T07:09:54.499896+0000","last_clean":"2024-09-14T07:09:54.499896+0000","last_became_active":"2024-09-14T07:09:14.796081+0000","last_became_peered":"2024-09-14T07:09:14.796081+0000","last_unstale":"2024-09-14T07:09:54.499896+0000","last_undegraded":"2024-09-14T07:09:54.499896+0000","last_fullsized":"2024-09-14T07:09:54.499896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:02:32.749998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":58557771,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":8398,"num_read_kb":29508,"num_write":1176,"num_write_kb":11644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":17139225,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"113'283","reported_seq":13423,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.249916+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:09:55.249916+0000","last_peered":"2024-09-14T07:09:55.249916+0000","last_clean":"2024-09-14T07:09:55.249916+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:09:55.249916+0000","last_undegraded":"2024-09-14T07:09:55.249916+0000","last_fullsized":"2024-09-14T07:09:55.249916+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:17.094132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":50191360,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":14777,"num_read_kb":26750,"num_write":203,"num_write_kb":6237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"114'191","reported_seq":2901,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501274+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:54.501274+0000","last_peered":"2024-09-14T07:09:54.501274+0000","last_clean":"2024-09-14T07:09:54.501274+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:54.501274+0000","last_undegraded":"2024-09-14T07:09:54.501274+0000","last_fullsized":"2024-09-14T07:09:54.501274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:46.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":27958315,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":2639,"num_read_kb":13197,"num_write":195,"num_write_kb":2831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"114'252","reported_seq":15193,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.246305+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:55.246305+0000","last_peered":"2024-09-14T07:09:55.246305+0000","last_clean":"2024-09-14T07:09:55.246305+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:55.246305+0000","last_undegraded":"2024-09-14T07:09:55.246305+0000","last_fullsized":"2024-09-14T07:09:55.246305+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:14.130306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":42513061,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":15815,"num_read_kb":30148,"num_write":209,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"114'299","reported_seq":7181,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.231154+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:55.231154+0000","last_peered":"2024-09-14T07:09:55.231154+0000","last_clean":"2024-09-14T07:09:55.231154+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:55.231154+0000","last_undegraded":"2024-09-14T07:09:55.231154+0000","last_fullsized":"2024-09-14T07:09:55.231154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:30.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":31893641,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":7510,"num_read_kb":12382,"num_write":459,"num_write_kb":16316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"114'189","reported_seq":6024,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522600+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:09:54.522600+0000","last_peered":"2024-09-14T07:09:54.522600+0000","last_clean":"2024-09-14T07:09:54.522600+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:09:54.522600+0000","last_undegraded":"2024-09-14T07:09:54.522600+0000","last_fullsized":"2024-09-14T07:09:54.522600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:52.464196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":51962130,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":5305,"num_read_kb":17493,"num_write":295,"num_write_kb":15778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"109'306","reported_seq":7281,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500987+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:54.500987+0000","last_peered":"2024-09-14T07:09:54.500987+0000","last_clean":"2024-09-14T07:09:54.500987+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:54.500987+0000","last_undegraded":"2024-09-14T07:09:54.500987+0000","last_fullsized":"2024-09-14T07:09:54.500987+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:07:33.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":59253391,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":7719,"num_read_kb":17127,"num_write":243,"num_write_kb":7138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"117'296","reported_seq":5885,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501120+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:09:54.501120+0000","last_peered":"2024-09-14T07:09:54.501120+0000","last_clean":"2024-09-14T07:09:54.501120+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:09:54.501120+0000","last_undegraded":"2024-09-14T07:09:54.501120+0000","last_fullsized":"2024-09-14T07:09:54.501120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:22.468213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":55804416,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":6428,"num_read_kb":20704,"num_write":201,"num_write_kb":6545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"113'202","reported_seq":2924,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.216581+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:55.216581+0000","last_peered":"2024-09-14T07:09:55.216581+0000","last_clean":"2024-09-14T07:09:55.216581+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:55.216581+0000","last_undegraded":"2024-09-14T07:09:55.216581+0000","last_fullsized":"2024-09-14T07:09:55.216581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:48.925480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":65290804,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":4422,"num_read_kb":38443,"num_write":206,"num_write_kb":4783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1627752035,"num_objects":871,"num_object_clones":143,"num_object_copies":1742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":871,"num_whiteouts":234,"num_read":249650,"num_read_kb":766430,"num_write":11566,"num_write_kb":257838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":127562704,"num_keys_recovered":288,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33480704,"data_stored":308175968,"data_compressed":13066350,"data_compressed_allocated":27545600,"data_compressed_original":304943616,"omap_allocated":0,"internal_metadata":0},"log_size":9503,"ondisk_log_size":9503,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39312,"kb_used_data":8128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332528,"statfs":{"total":96636764160,"available":96596508672,"internally_reserved":0,"allocated":8323072,"data_stored":56827386,"data_compressed":3012761,"data_compressed_allocated":5742592,"data_compressed_original":55636480,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,44,129],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607585,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36080,"kb_used_data":4884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335760,"statfs":{"total":96636764160,"available":96599818240,"internally_reserved":0,"allocated":5001216,"data_stored":21425508,"data_compressed":1716871,"data_compressed_allocated":2838528,"data_compressed_original":20449280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,129,70],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607584,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38160,"kb_used_data":6968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333680,"statfs":{"total":96636764160,"available":96597688320,"internally_reserved":0,"allocated":7135232,"data_stored":56123881,"data_compressed":2213852,"data_compressed_allocated":4755456,"data_compressed_original":54997504,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607586,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":6408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":6561792,"data_stored":47759640,"data_compressed":1994534,"data_compressed_allocated":4190208,"data_compressed_original":46702080,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607586,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":6432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":6586368,"data_stored":59848325,"data_compressed":1668385,"data_compressed_allocated":4329472,"data_compressed_original":58853376,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607586,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35116,"kb_used_data":8080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336724,"statfs":{"total":96636764160,"available":96600805376,"internally_reserved":0,"allocated":8273920,"data_stored":70728270,"data_compressed":2472897,"data_compressed_allocated":5763072,"data_compressed_original":69468160,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6860800,"data_stored":69582722,"data_compressed":2466005,"data_compressed_allocated":5726208,"data_compressed_original":68878336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5210112,"data_stored":59293126,"data_compressed":1668385,"data_compressed_allocated":4329472,"data_compressed_original":58853376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5177344,"data_stored":47199490,"data_compressed":1994534,"data_compressed_allocated":4190208,"data_compressed_original":46702080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5693440,"data_stored":54973363,"data_compressed":2207794,"data_compressed_allocated":4718592,"data_compressed_original":54424064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6914048,"data_stored":56256958,"data_compressed":3012761,"data_compressed_allocated":5742592,"data_compressed_original":55636480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3624960,"data_stored":20870309,"data_compressed":1716871,"data_compressed_allocated":2838528,"data_compressed_original":20449280,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:57.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:09:57.999 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:09:57.999 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:58.000 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:57.999+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:58.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:09:58.073 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:58.071+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:58.174 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:09:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:09:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:09:58.175 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:58.175 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:58.171+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:58.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:09:58.265 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:09:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:09:58.274 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:58.271+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:58.283 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-14T07:09:57.521418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1628342422,"num_objects":875,"num_object_clones":143,"num_object_copies":1750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":875,"num_whiteouts":234,"num_read":249716,"num_read_kb":766485,"num_write":11669,"num_write_kb":259468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1004,"num_bytes_recovered":128743440,"num_keys_recovered":288,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9578,"ondisk_log_size":9578,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":219732,"kb_used_data":40900,"kb_used_omap":9,"kb_used_meta":178678,"kb_avail":566011308,"statfs":{"total":579820584960,"available":579595579392,"internally_reserved":0,"allocated":41881600,"data_stored":312713010,"data_compressed":13079300,"data_compressed_allocated":27619328,"data_compressed_original":306106880,"omap_allocated":9568,"internal_metadata":182966944},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,129,114,129],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25649314,"num_objects":30,"num_object_clones":14,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":18238,"num_read_kb":26026,"num_write":252,"num_write_kb":1078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025037"},"pg_stats":[{"pgid":"3.1f","version":"113'148","reported_seq":5065,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.018441+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:09:55.018441+0000","last_peered":"2024-09-14T07:09:55.018441+0000","last_clean":"2024-09-14T07:09:55.018441+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:09:55.018441+0000","last_undegraded":"2024-09-14T07:09:55.018441+0000","last_fullsized":"2024-09-14T07:09:55.018441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:37.693843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":69978128,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":4454,"num_read_kb":43252,"num_write":232,"num_write_kb":10177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"113'450","reported_seq":8421,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.225739+0000","last_change":"2024-09-14T07:07:53.850731+0000","last_active":"2024-09-14T07:09:55.225739+0000","last_peered":"2024-09-14T07:09:55.225739+0000","last_clean":"2024-09-14T07:09:55.225739+0000","last_became_active":"2024-09-14T07:07:53.833663+0000","last_became_peered":"2024-09-14T07:07:53.833663+0000","last_unstale":"2024-09-14T07:09:55.225739+0000","last_undegraded":"2024-09-14T07:09:55.225739+0000","last_fullsized":"2024-09-14T07:09:55.225739+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:08:04.096796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":66759046,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":9323,"num_read_kb":40106,"num_write":734,"num_write_kb":13123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"114'600","reported_seq":8898,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522670+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:09:54.522670+0000","last_peered":"2024-09-14T07:09:54.522670+0000","last_clean":"2024-09-14T07:09:54.522670+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:09:54.522670+0000","last_undegraded":"2024-09-14T07:09:54.522670+0000","last_fullsized":"2024-09-14T07:09:54.522670+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:06.985688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":10296420,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":11640,"num_read_kb":20380,"num_write":411,"num_write_kb":2962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"114'176","reported_seq":13801,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.243958+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:09:55.243958+0000","last_peered":"2024-09-14T07:09:55.243958+0000","last_clean":"2024-09-14T07:09:55.243958+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:09:55.243958+0000","last_undegraded":"2024-09-14T07:09:55.243958+0000","last_fullsized":"2024-09-14T07:09:55.243958+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:12:37.762139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":37774900,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":13469,"num_read_kb":20176,"num_write":191,"num_write_kb":4683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"109'377","reported_seq":7702,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.018654+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:09:55.018654+0000","last_peered":"2024-09-14T07:09:55.018654+0000","last_clean":"2024-09-14T07:09:55.018654+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:09:55.018654+0000","last_undegraded":"2024-09-14T07:09:55.018654+0000","last_fullsized":"2024-09-14T07:09:55.018654+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:22.272156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":102915259,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":8067,"num_read_kb":21163,"num_write":512,"num_write_kb":10477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"114'217","reported_seq":27104,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:57.082789+0000","last_change":"2024-09-14T07:08:33.769426+0000","last_active":"2024-09-14T07:09:57.082789+0000","last_peered":"2024-09-14T07:09:57.082789+0000","last_clean":"2024-09-14T07:09:57.082789+0000","last_became_active":"2024-09-14T07:08:33.768735+0000","last_became_peered":"2024-09-14T07:08:33.768735+0000","last_unstale":"2024-09-14T07:09:57.082789+0000","last_undegraded":"2024-09-14T07:09:57.082789+0000","last_fullsized":"2024-09-14T07:09:57.082789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:16:36.796265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":81449645,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":27691,"num_read_kb":46889,"num_write":260,"num_write_kb":10858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":16841766,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"114'179","reported_seq":5340,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499719+0000","last_change":"2024-09-14T07:08:31.198632+0000","last_active":"2024-09-14T07:09:54.499719+0000","last_peered":"2024-09-14T07:09:54.499719+0000","last_clean":"2024-09-14T07:09:54.499719+0000","last_became_active":"2024-09-14T07:08:31.113201+0000","last_became_peered":"2024-09-14T07:08:31.113201+0000","last_unstale":"2024-09-14T07:09:54.499719+0000","last_undegraded":"2024-09-14T07:09:54.499719+0000","last_fullsized":"2024-09-14T07:09:54.499719+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:26:24.800199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":30128251,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":5081,"num_read_kb":12880,"num_write":198,"num_write_kb":5982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2122828,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"114'169","reported_seq":2603,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.223771+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:09:55.223771+0000","last_peered":"2024-09-14T07:09:55.223771+0000","last_clean":"2024-09-14T07:09:55.223771+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:09:55.223771+0000","last_undegraded":"2024-09-14T07:09:55.223771+0000","last_fullsized":"2024-09-14T07:09:55.223771+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:47:37.142427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":76865736,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":2225,"num_read_kb":6268,"num_write":287,"num_write_kb":13843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"109'116","reported_seq":1926,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522837+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:09:54.522837+0000","last_peered":"2024-09-14T07:09:54.522837+0000","last_clean":"2024-09-14T07:09:54.522837+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:09:54.522837+0000","last_undegraded":"2024-09-14T07:09:54.522837+0000","last_fullsized":"2024-09-14T07:09:54.522837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:18:52.224542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":37352486,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1465,"num_read_kb":9238,"num_write":132,"num_write_kb":2767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"109'252","reported_seq":7108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522737+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:09:54.522737+0000","last_peered":"2024-09-14T07:09:54.522737+0000","last_clean":"2024-09-14T07:09:54.522737+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:09:54.522737+0000","last_undegraded":"2024-09-14T07:09:54.522737+0000","last_fullsized":"2024-09-14T07:09:54.522737+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:09.242953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":48148006,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":7629,"num_read_kb":32754,"num_write":224,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"112'255","reported_seq":4435,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.228808+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:09:55.228808+0000","last_peered":"2024-09-14T07:09:55.228808+0000","last_clean":"2024-09-14T07:09:55.228808+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:09:55.228808+0000","last_undegraded":"2024-09-14T07:09:55.228808+0000","last_fullsized":"2024-09-14T07:09:55.228808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:50:58.984154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":67224625,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":5248,"num_read_kb":38739,"num_write":320,"num_write_kb":13909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"114'509","reported_seq":8721,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500997+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:09:54.500997+0000","last_peered":"2024-09-14T07:09:54.500997+0000","last_clean":"2024-09-14T07:09:54.500997+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:09:54.500997+0000","last_undegraded":"2024-09-14T07:09:54.500997+0000","last_fullsized":"2024-09-14T07:09:54.500997+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:20:28.164748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":59498099,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":9948,"num_read_kb":31183,"num_write":715,"num_write_kb":13153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":229,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501334+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:09:54.501334+0000","last_peered":"2024-09-14T07:09:54.501334+0000","last_clean":"2024-09-14T07:09:54.501334+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:09:54.501334+0000","last_undegraded":"2024-09-14T07:09:54.501334+0000","last_fullsized":"2024-09-14T07:09:54.501334+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'428","reported_seq":9319,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.018233+0000","last_change":"2024-09-14T07:09:14.793462+0000","last_active":"2024-09-14T07:09:55.018233+0000","last_peered":"2024-09-14T07:09:55.018233+0000","last_clean":"2024-09-14T07:09:55.018233+0000","last_became_active":"2024-09-14T07:09:14.787124+0000","last_became_peered":"2024-09-14T07:09:14.787124+0000","last_unstale":"2024-09-14T07:09:55.018233+0000","last_undegraded":"2024-09-14T07:09:55.018233+0000","last_fullsized":"2024-09-14T07:09:55.018233+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:54:53.102179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":21480210,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":6,"num_read":11359,"num_read_kb":21400,"num_write":511,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":1461720,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":217,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499856+0000","last_change":"2024-09-14T07:07:38.032413+0000","last_active":"2024-09-14T07:09:54.499856+0000","last_peered":"2024-09-14T07:09:54.499856+0000","last_clean":"2024-09-14T07:09:54.499856+0000","last_became_active":"2024-09-14T07:07:36.017495+0000","last_became_peered":"2024-09-14T07:07:36.017495+0000","last_unstale":"2024-09-14T07:09:54.499856+0000","last_undegraded":"2024-09-14T07:09:54.499856+0000","last_fullsized":"2024-09-14T07:09:54.499856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:19.394599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001423246,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"113'205","reported_seq":4289,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.540585+0000","last_change":"2024-09-14T07:07:53.840299+0000","last_active":"2024-09-14T07:09:54.540585+0000","last_peered":"2024-09-14T07:09:54.540585+0000","last_clean":"2024-09-14T07:09:54.540585+0000","last_became_active":"2024-09-14T07:07:53.837105+0000","last_became_peered":"2024-09-14T07:07:53.837105+0000","last_unstale":"2024-09-14T07:09:54.540585+0000","last_undegraded":"2024-09-14T07:09:54.540585+0000","last_fullsized":"2024-09-14T07:09:54.540585+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:36:39.915820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":33995806,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":4605,"num_read_kb":13735,"num_write":248,"num_write_kb":2656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":229,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501182+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:09:54.501182+0000","last_peered":"2024-09-14T07:09:54.501182+0000","last_clean":"2024-09-14T07:09:54.501182+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:09:54.501182+0000","last_undegraded":"2024-09-14T07:09:54.501182+0000","last_fullsized":"2024-09-14T07:09:54.501182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"113'460","reported_seq":4675,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.231694+0000","last_change":"2024-09-14T07:07:53.867771+0000","last_active":"2024-09-14T07:09:55.231694+0000","last_peered":"2024-09-14T07:09:55.231694+0000","last_clean":"2024-09-14T07:09:55.231694+0000","last_became_active":"2024-09-14T07:07:53.842818+0000","last_became_peered":"2024-09-14T07:07:53.842818+0000","last_unstale":"2024-09-14T07:09:55.231694+0000","last_undegraded":"2024-09-14T07:09:55.231694+0000","last_fullsized":"2024-09-14T07:09:55.231694+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:16:38.943523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":4813879,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":6364,"num_read_kb":9719,"num_write":667,"num_write_kb":2103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":229,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500020+0000","last_change":"2024-09-14T07:09:11.349379+0000","last_active":"2024-09-14T07:09:54.500020+0000","last_peered":"2024-09-14T07:09:54.500020+0000","last_clean":"2024-09-14T07:09:54.500020+0000","last_became_active":"2024-09-14T07:09:11.348941+0000","last_became_peered":"2024-09-14T07:09:11.348941+0000","last_unstale":"2024-09-14T07:09:54.500020+0000","last_undegraded":"2024-09-14T07:09:54.500020+0000","last_fullsized":"2024-09-14T07:09:54.500020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:20:39.931750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"114'529","reported_seq":2629,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522789+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:09:54.522789+0000","last_peered":"2024-09-14T07:09:54.522789+0000","last_clean":"2024-09-14T07:09:54.522789+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:09:54.522789+0000","last_undegraded":"2024-09-14T07:09:54.522789+0000","last_fullsized":"2024-09-14T07:09:54.522789+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:10:04.835880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":13788461,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":3,"num_read":4187,"num_read_kb":16563,"num_write":919,"num_write_kb":4981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"113'116","reported_seq":6299,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:56.523828+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:09:56.523828+0000","last_peered":"2024-09-14T07:09:56.523828+0000","last_clean":"2024-09-14T07:09:56.523828+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:09:56.523828+0000","last_undegraded":"2024-09-14T07:09:56.523828+0000","last_fullsized":"2024-09-14T07:09:56.523828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:09:24.740371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":63791616,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":5613,"num_read_kb":24884,"num_write":197,"num_write_kb":12074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":219,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522836+0000","last_change":"2024-09-14T07:07:38.038020+0000","last_active":"2024-09-14T07:09:54.522836+0000","last_peered":"2024-09-14T07:09:54.522836+0000","last_clean":"2024-09-14T07:09:54.522836+0000","last_became_active":"2024-09-14T07:07:36.017310+0000","last_became_peered":"2024-09-14T07:07:36.017310+0000","last_unstale":"2024-09-14T07:09:54.522836+0000","last_undegraded":"2024-09-14T07:09:54.522836+0000","last_fullsized":"2024-09-14T07:09:54.522836+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:53.007336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"113'272","reported_seq":3521,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499457+0000","last_change":"2024-09-14T07:07:53.843804+0000","last_active":"2024-09-14T07:09:54.499457+0000","last_peered":"2024-09-14T07:09:54.499457+0000","last_clean":"2024-09-14T07:09:54.499457+0000","last_became_active":"2024-09-14T07:07:53.842798+0000","last_became_peered":"2024-09-14T07:07:53.842798+0000","last_unstale":"2024-09-14T07:09:54.499457+0000","last_undegraded":"2024-09-14T07:09:54.499457+0000","last_fullsized":"2024-09-14T07:09:54.499457+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:11.014271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":27308868,"num_objects":29,"num_object_clones":4,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":6,"num_read":5320,"num_read_kb":11387,"num_write":196,"num_write_kb":2794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":193,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.540321+0000","last_change":"2024-09-14T07:07:53.839290+0000","last_active":"2024-09-14T07:09:54.540321+0000","last_peered":"2024-09-14T07:09:54.540321+0000","last_clean":"2024-09-14T07:09:54.540321+0000","last_became_active":"2024-09-14T07:07:53.838346+0000","last_became_peered":"2024-09-14T07:07:53.838346+0000","last_unstale":"2024-09-14T07:09:54.540321+0000","last_undegraded":"2024-09-14T07:09:54.540321+0000","last_fullsized":"2024-09-14T07:09:54.540321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:41.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"119'191","reported_seq":9496,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:56.523896+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:09:56.523896+0000","last_peered":"2024-09-14T07:09:56.523896+0000","last_clean":"2024-09-14T07:09:56.523896+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:09:56.523896+0000","last_undegraded":"2024-09-14T07:09:56.523896+0000","last_fullsized":"2024-09-14T07:09:56.523896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:12.502326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":19655209,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":9101,"num_read_kb":9780,"num_write":172,"num_write_kb":1964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":217,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500844+0000","last_change":"2024-09-14T07:07:38.031792+0000","last_active":"2024-09-14T07:09:54.500844+0000","last_peered":"2024-09-14T07:09:54.500844+0000","last_clean":"2024-09-14T07:09:54.500844+0000","last_became_active":"2024-09-14T07:07:36.015960+0000","last_became_peered":"2024-09-14T07:07:36.015960+0000","last_unstale":"2024-09-14T07:09:54.500844+0000","last_undegraded":"2024-09-14T07:09:54.500844+0000","last_fullsized":"2024-09-14T07:09:54.500844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:08:08.251047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"113'189","reported_seq":5044,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499566+0000","last_change":"2024-09-14T07:07:49.813279+0000","last_active":"2024-09-14T07:09:54.499566+0000","last_peered":"2024-09-14T07:09:54.499566+0000","last_clean":"2024-09-14T07:09:54.499566+0000","last_became_active":"2024-09-14T07:07:47.786516+0000","last_became_peered":"2024-09-14T07:07:47.786516+0000","last_unstale":"2024-09-14T07:09:54.499566+0000","last_undegraded":"2024-09-14T07:09:54.499566+0000","last_fullsized":"2024-09-14T07:09:54.499566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:46:40.422390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":50891808,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":4675,"num_read_kb":7307,"num_write":174,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":238,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522679+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:09:54.522679+0000","last_peered":"2024-09-14T07:09:54.522679+0000","last_clean":"2024-09-14T07:09:54.522679+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:09:54.522679+0000","last_undegraded":"2024-09-14T07:09:54.522679+0000","last_fullsized":"2024-09-14T07:09:54.522679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":332,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500973+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:09:54.500973+0000","last_peered":"2024-09-14T07:09:54.500973+0000","last_clean":"2024-09-14T07:09:54.500973+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:09:54.500973+0000","last_undegraded":"2024-09-14T07:09:54.500973+0000","last_fullsized":"2024-09-14T07:09:54.500973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"114'235","reported_seq":7983,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.227010+0000","last_change":"2024-09-14T07:07:53.855433+0000","last_active":"2024-09-14T07:09:55.227010+0000","last_peered":"2024-09-14T07:09:55.227010+0000","last_clean":"2024-09-14T07:09:55.227010+0000","last_became_active":"2024-09-14T07:07:53.837427+0000","last_became_peered":"2024-09-14T07:07:53.837427+0000","last_unstale":"2024-09-14T07:09:55.227010+0000","last_undegraded":"2024-09-14T07:09:55.227010+0000","last_fullsized":"2024-09-14T07:09:55.227010+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:33:42.242093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":144767060,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":5,"num_read":7483,"num_read_kb":59764,"num_write":383,"num_write_kb":25537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"119'430","reported_seq":5082,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-14T07:09:56.524089+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:09:56.524089+0000","last_peered":"2024-09-14T07:09:56.524089+0000","last_clean":"2024-09-14T07:09:56.524089+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:09:56.524089+0000","last_undegraded":"2024-09-14T07:09:56.524089+0000","last_fullsized":"2024-09-14T07:09:56.524089+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:37:26.541095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":82193820,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":6168,"num_read_kb":39944,"num_write":494,"num_write_kb":16026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"114'196","reported_seq":5860,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501183+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:09:54.501183+0000","last_peered":"2024-09-14T07:09:54.501183+0000","last_clean":"2024-09-14T07:09:54.501183+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:09:54.501183+0000","last_undegraded":"2024-09-14T07:09:54.501183+0000","last_fullsized":"2024-09-14T07:09:54.501183+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:02:30.086221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":33249808,"num_objects":28,"num_object_clones":7,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":5522,"num_read_kb":23167,"num_write":202,"num_write_kb":2786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"114'786","reported_seq":5716,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.499896+0000","last_change":"2024-09-14T07:09:14.798144+0000","last_active":"2024-09-14T07:09:54.499896+0000","last_peered":"2024-09-14T07:09:54.499896+0000","last_clean":"2024-09-14T07:09:54.499896+0000","last_became_active":"2024-09-14T07:09:14.796081+0000","last_became_peered":"2024-09-14T07:09:14.796081+0000","last_unstale":"2024-09-14T07:09:54.499896+0000","last_undegraded":"2024-09-14T07:09:54.499896+0000","last_fullsized":"2024-09-14T07:09:54.499896+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:02:32.749998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":58557771,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":8398,"num_read_kb":29508,"num_write":1176,"num_write_kb":11644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":17139225,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"113'283","reported_seq":13423,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.249916+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:09:55.249916+0000","last_peered":"2024-09-14T07:09:55.249916+0000","last_clean":"2024-09-14T07:09:55.249916+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:09:55.249916+0000","last_undegraded":"2024-09-14T07:09:55.249916+0000","last_fullsized":"2024-09-14T07:09:55.249916+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:17.094132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":50191360,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":4,"num_read":14777,"num_read_kb":26750,"num_write":203,"num_write_kb":6237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"114'191","reported_seq":2901,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501274+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:09:54.501274+0000","last_peered":"2024-09-14T07:09:54.501274+0000","last_clean":"2024-09-14T07:09:54.501274+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:09:54.501274+0000","last_undegraded":"2024-09-14T07:09:54.501274+0000","last_fullsized":"2024-09-14T07:09:54.501274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:46.306835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":27958315,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":2639,"num_read_kb":13197,"num_write":195,"num_write_kb":2831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"114'252","reported_seq":15193,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.246305+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:09:55.246305+0000","last_peered":"2024-09-14T07:09:55.246305+0000","last_clean":"2024-09-14T07:09:55.246305+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:09:55.246305+0000","last_undegraded":"2024-09-14T07:09:55.246305+0000","last_fullsized":"2024-09-14T07:09:55.246305+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:14.130306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":42513061,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":15815,"num_read_kb":30148,"num_write":209,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"114'299","reported_seq":7181,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.231154+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:09:55.231154+0000","last_peered":"2024-09-14T07:09:55.231154+0000","last_clean":"2024-09-14T07:09:55.231154+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:09:55.231154+0000","last_undegraded":"2024-09-14T07:09:55.231154+0000","last_fullsized":"2024-09-14T07:09:55.231154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:30.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":31893641,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":7510,"num_read_kb":12382,"num_write":459,"num_write_kb":16316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"114'189","reported_seq":6024,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.522600+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:09:54.522600+0000","last_peered":"2024-09-14T07:09:54.522600+0000","last_clean":"2024-09-14T07:09:54.522600+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:09:54.522600+0000","last_undegraded":"2024-09-14T07:09:54.522600+0000","last_fullsized":"2024-09-14T07:09:54.522600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:52.464196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":51962130,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":9,"num_read":5305,"num_read_kb":17493,"num_write":295,"num_write_kb":15778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"109'306","reported_seq":7281,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.500987+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:09:54.500987+0000","last_peered":"2024-09-14T07:09:54.500987+0000","last_clean":"2024-09-14T07:09:54.500987+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:09:54.500987+0000","last_undegraded":"2024-09-14T07:09:54.500987+0000","last_fullsized":"2024-09-14T07:09:54.500987+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:07:33.567422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":59253391,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":7719,"num_read_kb":17127,"num_write":243,"num_write_kb":7138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"117'296","reported_seq":5885,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:54.501120+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:09:54.501120+0000","last_peered":"2024-09-14T07:09:54.501120+0000","last_clean":"2024-09-14T07:09:54.501120+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:09:54.501120+0000","last_undegraded":"2024-09-14T07:09:54.501120+0000","last_fullsized":"2024-09-14T07:09:54.501120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:22.468213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":55804416,"num_objects":29,"num_object_clones":2,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":6428,"num_read_kb":20704,"num_write":201,"num_write_kb":6545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"113'202","reported_seq":2924,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-14T07:09:55.216581+0000","last_change":"2024-09-14T07:08:38.210607+0000","last_active":"2024-09-14T07:09:55.216581+0000","last_peered":"2024-09-14T07:09:55.216581+0000","last_clean":"2024-09-14T07:09:55.216581+0000","last_became_active":"2024-09-14T07:08:38.121503+0000","last_became_peered":"2024-09-14T07:08:38.121503+0000","last_unstale":"2024-09-14T07:09:55.216581+0000","last_undegraded":"2024-09-14T07:09:55.216581+0000","last_fullsized":"2024-09-14T07:09:55.216581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:48.925480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":65290804,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":4422,"num_read_kb":38443,"num_write":206,"num_write_kb":4783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":3257872,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1627752035,"num_objects":871,"num_object_clones":143,"num_object_copies":1742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":871,"num_whiteouts":234,"num_read":249650,"num_read_kb":766430,"num_write":11566,"num_write_kb":257838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":127562704,"num_keys_recovered":288,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33480704,"data_stored":308175968,"data_compressed":13066350,"data_compressed_allocated":27545600,"data_compressed_original":304943616,"omap_allocated":0,"internal_metadata":0},"log_size":9503,"ondisk_log_size":9503,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39312,"kb_used_data":8128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332528,"statfs":{"total":96636764160,"available":96596508672,"internally_reserved":0,"allocated":8323072,"data_stored":56827386,"data_compressed":3012761,"data_compressed_allocated":5742592,"data_compressed_original":55636480,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,44,129],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607585,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36080,"kb_used_data":4884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335760,"statfs":{"total":96636764160,"available":96599818240,"internally_reserved":0,"allocated":5001216,"data_stored":21425508,"data_compressed":1716871,"data_compressed_allocated":2838528,"data_compressed_original":20449280,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,129,70],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607584,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38160,"kb_used_data":6968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333680,"statfs":{"total":96636764160,"available":96597688320,"internally_reserved":0,"allocated":7135232,"data_stored":56123881,"data_compressed":2213852,"data_compressed_allocated":4755456,"data_compressed_original":54997504,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607586,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":6408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":6561792,"data_stored":47759640,"data_compressed":1994534,"data_compressed_allocated":4190208,"data_compressed_original":46702080,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607586,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":6432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":6586368,"data_stored":59848325,"data_compressed":1668385,"data_compressed_allocated":4329472,"data_compressed_original":58853376,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607586,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35116,"kb_used_data":8080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336724,"statfs":{"total":96636764160,"available":96600805376,"internally_reserved":0,"allocated":8273920,"data_stored":70728270,"data_compressed":2472897,"data_compressed_allocated":5763072,"data_compressed_original":69468160,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6860800,"data_stored":69582722,"data_compressed":2466005,"data_compressed_allocated":5726208,"data_compressed_original":68878336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5210112,"data_stored":59293126,"data_compressed":1668385,"data_compressed_allocated":4329472,"data_compressed_original":58853376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5177344,"data_stored":47199490,"data_compressed":1994534,"data_compressed_allocated":4190208,"data_compressed_original":46702080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5693440,"data_stored":54973363,"data_compressed":2207794,"data_compressed_allocated":4718592,"data_compressed_original":54424064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6914048,"data_stored":56256958,"data_compressed":3012761,"data_compressed_allocated":5742592,"data_compressed_original":55636480,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3624960,"data_stored":20870309,"data_compressed":1716871,"data_compressed_allocated":2838528,"data_compressed_original":20449280,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:09:58.285 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T07:09:58.285 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:09:58.286 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [1, 3, 5, 2, 4, 0] 2024-09-14T07:09:58.286 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-14T07:09:58.319 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:09:58.319 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:09:58.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:09:58.319 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.017428498, 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:58.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.972438192, 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:09:58.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 96.670505735999996, 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:09:58.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:09:58.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:09:58.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 96.349178002000002, 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:09:58.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:09:58.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.912049603, 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:09:58.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:09:58.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.912038944999999, 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.912023931, 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:09:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.912013016000003, 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:09:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:09:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.912001506000003, 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:09:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:09:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:09:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.911988962999999, 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:09:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905152960999999, 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:09:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:09:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905142367000003, 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:09:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:09:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905094585999997, 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:09:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:09:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905081119000002, 2024-09-14T07:09:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905070147000004, 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:09:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:09:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:09:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905059385000001, 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:09:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905043331000002, 2024-09-14T07:09:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905026759999998, 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:09:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:09:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905014983999997, 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.905004306999999, 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:09:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:09:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:09:58.413 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:09:58.415 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:09:58.415 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:58.411+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:58.516 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:58.515+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:58.538 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:09:58.549 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:58.549 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:09:58.549 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:58.565 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:09:58.603 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:09:58.603+0000 7f1e6b537640 -1 mon.a@0(leader).osd e121 definitely_dead 0 2024-09-14T07:09:58.652 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:58.651+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:58.717 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:58.715+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:58.733 INFO:teuthology.orchestra.run.smithi145.stdout:leaving image image_client.0-clone30 intact 2024-09-14T07:09:58.737 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:09:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:09:58.737 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:09:58.737 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:58.751 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:09:58.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:09:58.844 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:09:58.843+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:09:58.919 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:58.915+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:58.927 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:09:58.928 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:09:58.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:09:58.928 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:09:58.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.928 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:09:58.928 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 146.624685672, 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:09:58.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:09:58.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:09:58.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:09:58.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:09:58.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:09:58.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:09:58.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:58.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.730692488000003, 2024-09-14T07:09:58.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:09:58.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:09:58.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:09:58.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:09:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:09:58.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.730572609000006, 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.946 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:09:58.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:09:58.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:09:58.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.713347489, 2024-09-14T07:09:58.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.040753101999996, 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:09:58.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:09:58.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:09:58.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:09:58.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266711235999999, 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:09:58.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:09:58.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:09:58.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266679007999997, 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:09:58.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266656992999998, 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:58.979 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266636058000003, 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:09:58.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:09:58.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266615047999998, 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:09:58.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266581348000003, 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:09:58.988 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:09:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:09:58.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.992 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266560705000003, 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.993 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:09:58.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:09:58.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:58.997 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266315487999996, 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:58.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:58.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266283768999998, 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.002 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:09:59.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:09:59.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:09:59.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266262009999998, 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:59.006 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:59.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:09:59.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.266229930999998, 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.011 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.014 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.265058725000003, 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:59.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:09:59.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.265035761999997, 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:09:59.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:59.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:09:59.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:09:59.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:09:59.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:09:59.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:09:59.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.265014471999997, 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:09:59.026 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:59.027 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:59.027 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:59.027 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:59.027 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:59.027 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:59.027 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:09:59.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:09:59.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:09:59.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.031 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.264994248999997, 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:09:59.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:09:59.037 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:09:59.037 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:09:59.038 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:09:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:09:59.039 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:09:59.039 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:09:59.039 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:09:59.040 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:59.035+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:59.040 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:09:59.040 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:09:59.054 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:09:59.054 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:09:59.075 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:09:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:09:59.096 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:09:59.096 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:09:59.155 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:59.155+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:59.211 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:59.211 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:59.226 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:09:59.239 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:59.239+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:59.340 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:59.339+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:59.353 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:59.353 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:59.367 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:09:59.440 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:59.439+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:59.490 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T07:09:59.498 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:59.498 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:59.514 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:09:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.1. 2024-09-14T07:09:59.553 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:09:59.543+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:09:59.624 INFO:teuthology.orchestra.run.smithi145.stdout:4245 trunc from 0xce4e200 to 0x9270400 2024-09-14T07:09:59.642 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:09:59.639+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:09:59.678 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:59.678 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:59.692 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:09:59.743 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:09:59.743+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:09:59.815 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:59.816 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:59.830 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:09:59.876 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:09:59.875+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:09:59.946 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:09:59.943+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:09:59.953 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:09:59.953 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:09:59.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:00.047 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:00.043+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:00.105 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:00.105 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:00.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:00.163 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:00.163+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:00.249 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:00.247+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:00.282 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:00.282 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:00.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:00.387 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:00.387+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:00.450 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:00.447+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:00.467 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:00.468 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:00.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:00.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:10:00.567 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:00.567+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:00.652 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:00.651+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:00.661 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:00.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:00.753 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:00.751+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:00.835 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:00.836 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:00.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:00.853 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:00.851+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:00.954 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:00.951+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:00.969 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:00.969 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:00.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:01.071 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:01.071+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:01.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:01.154 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:01.154 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:01.155 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:01.155+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:01.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:01.264 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:01.259+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:01.296 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:01.296 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:01.319 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:01.319 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:01.341 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:01.341 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:01.349 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:01.349 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:01.349 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:01.357 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:01.355+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:01.360 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:01.361 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:01.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:01.458 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:01.455+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:01.545 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:01.545 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:01.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:01.545 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:01.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 104.24397636499999, 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.546 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:01.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:01.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.551 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 104.19898605900001, 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:01.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:01.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:01.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:01.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 99.897053603000003, 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 99.575725868999996, 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:01.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:01.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:01.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.138597470000001, 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:01.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:01.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:01.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.138586812, 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:01.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:01.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.138571798000001, 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:01.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:01.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.138560882999997, 2024-09-14T07:10:01.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:01.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.138549372999996, 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:01.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:01.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:01.593 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.13853683, 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:01.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:01.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:01.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2321 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 835584~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131700828, 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071862896999999995, 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2321 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412582+0000", 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412584+0000", 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.11e-06 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412587+0000", 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412589+0000", 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5320000000000001e-06 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484073+0000", 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071483987999999998 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484251+0000", 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017784100000000001 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484445+0000", 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019396699999999999 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131690233999997, 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:01.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.606 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131642452999998, 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:01.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131628986000003, 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:01.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:01.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131618013999997, 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:01.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:01.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131607252000002, 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:01.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:01.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131591198000002, 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:01.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:01.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131574626999999, 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:01.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:01.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:01.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:01.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131562850999998, 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:01.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.131552173999999, 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:01.645 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:01.645 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:01.646 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:10:01.646 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:01.648 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:01.643+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:01.653 INFO:teuthology.orchestra.run.smithi145.stdout:4246 read 0x2104c00 thru 0x21144bd (0xf8be bytes) 2024-09-14T07:10:01.746 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:01.743+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:01.797 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:01.797 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:01.797 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:01.798 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:01.814 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:01.861 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:01.859+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:01.941 INFO:teuthology.orchestra.run.smithi145.stdout:4247 write 0xb0ba400 thru 0xb0c75b0 (0xd1b1 bytes) 2024-09-14T07:10:01.941 INFO:teuthology.orchestra.run.smithi145.stdout:4248 write 0x36d400 thru 0x37751f (0xa120 bytes) 2024-09-14T07:10:01.941 INFO:teuthology.orchestra.run.smithi145.stdout:4249 read 0x5bb0600 thru 0x5bbeb28 (0xe529 bytes) 2024-09-14T07:10:01.949 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:01.950 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:01.950 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:01.950 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:01.950 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:01.957 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:01.951+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:01.966 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:02.005 INFO:teuthology.orchestra.run.smithi145.stdout:4251 trunc from 0xb0c7600 to 0x502b800 2024-09-14T07:10:02.057 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:02.055+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:02.165 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:02.151+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:02.166 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:02.167 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:02.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:02.167 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:02.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.167 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:02.167 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:02.168 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.85351513500001, 2024-09-14T07:10:02.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:02.168 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.168 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.168 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:02.168 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:02.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:02.169 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:02.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.169 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:02.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:02.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:02.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:02.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:02.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:02.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:02.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:02.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:02.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:02.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:02.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:02.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:02.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:02.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:02.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:02.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:02.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:02.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:02.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:02.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:02.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:02.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:02.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:02.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:02.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:02.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:02.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:02.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:02.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:02.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:02.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:02.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.190 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.190 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:02.190 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:02.190 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.959521950999999, 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:02.191 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:02.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.192 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:02.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:02.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:02.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:02.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:02.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:02.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:02.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:02.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:02.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:02.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:02.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:02.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:02.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:02.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:02.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:02.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:02.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:02.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:02.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:02.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:02.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:02.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:02.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:02.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:02.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:02.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:02.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:02.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:02.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:02.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:02.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:02.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:02.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:02.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:02.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:02.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:02.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:02.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:02.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:02.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.216 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:02.216 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:02.216 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.959402072000003, 2024-09-14T07:10:02.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:02.216 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.216 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.216 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:02.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:02.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:02.217 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:02.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.218 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:02.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:02.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:02.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:02.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:02.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:02.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:02.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:02.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:02.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:02.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:02.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:02.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:02.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:02.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:02.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:02.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:02.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:02.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:02.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:02.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:02.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:02.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:02.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:02.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:02.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:02.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:02.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:02.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:02.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:02.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:02.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:02.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:02.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:02.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:02.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:02.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:02.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:02.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:02.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:02.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.942176951999997, 2024-09-14T07:10:02.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:02.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:02.252 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.269582565, 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:02.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:02.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:02.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:02.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:02.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:02.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:02.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:02.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:02.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:02.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:02.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:02.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:02.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:02.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:02.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.269 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.270 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.270 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.270 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:02.271 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495540699000003, 2024-09-14T07:10:02.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:10:02.271 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.271 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.271 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.271 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.272 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.272 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.272 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:10:02.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.272 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:02.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:02.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:10:02.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:10:02.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:10:02.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:10:02.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:10:02.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:10:02.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:10:02.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:10:02.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:10:02.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:10:02.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:10:02.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:10:02.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:10:02.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:10:02.280 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.280 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.285 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.285 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.285 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:02.286 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495508471000001, 2024-09-14T07:10:02.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:02.286 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.286 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.286 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.287 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:02.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.287 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:02.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:02.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:02.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:02.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:02.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:02.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:02.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:02.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:02.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:02.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:02.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:02.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:02.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:02.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:02.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:02.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.300 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.301 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.301 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:02.301 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495486456000002, 2024-09-14T07:10:02.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:02.301 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.302 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.302 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.302 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:02.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.303 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:02.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:02.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:02.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:02.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:02.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:02.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:02.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:02.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:02.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:02.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:02.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:02.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:02.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:02.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:02.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:02.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.316 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.316 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:02.316 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495465521, 2024-09-14T07:10:02.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:02.317 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.317 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.317 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.317 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.317 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.317 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.318 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:02.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.318 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:02.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:02.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:02.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:02.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:02.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:02.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:02.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:02.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:02.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:02.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:02.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:02.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.330 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.330 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.330 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:02.330 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495444511000002, 2024-09-14T07:10:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.332 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:02.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.332 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:02.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:02.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:02.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:02.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:02.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:02.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:02.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:02.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:02.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:02.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:02.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:02.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:02.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:02.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:02.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:02.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.347 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.348 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.348 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:02.348 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495410810999999, 2024-09-14T07:10:02.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:02.349 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.349 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:02.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.350 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:02.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:02.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:02.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:02.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:02.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:02.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:02.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:02.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:02.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:02.356 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.356 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.356 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495390168, 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.357 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:02.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:02.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:02.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:02.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:02.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:02.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495144951, 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.365 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:02.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:02.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495113232000001, 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.369 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.374 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.374 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:02.374 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495091473000002, 2024-09-14T07:10:02.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:02.374 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.374 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:02.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.378 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.495059394000002, 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.379 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.383 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.493888188, 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.387 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.493865225, 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.388 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:02.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.493843935000001, 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.393 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:02.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 49.493823712000001, 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.397 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:02.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:02.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.401 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:02.404 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:02.405 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:02.399+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:02.408 INFO:teuthology.orchestra.run.smithi145.stdout:4252 punch from 0xda1800 to 0xda4000, (0x2800 bytes) 2024-09-14T07:10:02.408 INFO:teuthology.orchestra.run.smithi145.stdout:4255 punch from 0xff4600 to 0xffbc00, (0x7600 bytes) 2024-09-14T07:10:02.408 INFO:teuthology.orchestra.run.smithi145.stdout:4257 write 0x18f9200 thru 0x18f945e (0x25f bytes) 2024-09-14T07:10:02.408 INFO:teuthology.orchestra.run.smithi145.stdout:4258 read 0xe5cc00 thru 0xe68c7a (0xc07b bytes) 2024-09-14T07:10:02.408 INFO:teuthology.orchestra.run.smithi145.stdout:4259 punch from 0x752c00 to 0x761200, (0xe600 bytes) 2024-09-14T07:10:02.409 INFO:teuthology.orchestra.run.smithi145.stdout:4260 trunc from 0x502b800 to 0xced4a00 2024-09-14T07:10:02.409 INFO:teuthology.orchestra.run.smithi145.stdout:4261 read 0x7bad400 thru 0x7bb7b8b (0xa78c bytes) 2024-09-14T07:10:02.409 INFO:teuthology.orchestra.run.smithi145.stdout:4262 read 0xc7eba00 thru 0xc7ee4e4 (0x2ae5 bytes) 2024-09-14T07:10:02.409 INFO:teuthology.orchestra.run.smithi145.stdout:4264 read 0xc0b9a00 thru 0xc0c95ec (0xfbed bytes) 2024-09-14T07:10:02.409 INFO:teuthology.orchestra.run.smithi145.stdout:4265 write 0x9e68c00 thru 0x9e77697 (0xea98 bytes) 2024-09-14T07:10:02.409 INFO:teuthology.orchestra.run.smithi145.stdout:4269 read 0x84c7000 thru 0x84d289b (0xb89c bytes) 2024-09-14T07:10:02.410 INFO:teuthology.orchestra.run.smithi145.stdout:4272 read 0xa888e00 thru 0xa891144 (0x8345 bytes) 2024-09-14T07:10:02.410 INFO:teuthology.orchestra.run.smithi145.stdout:4274 read 0xb4e800 thru 0xb5ddac (0xf5ad bytes) 2024-09-14T07:10:02.410 INFO:teuthology.orchestra.run.smithi145.stdout:4277 read 0x7177c00 thru 0x717e1be (0x65bf bytes) 2024-09-14T07:10:02.410 INFO:teuthology.orchestra.run.smithi145.stdout:4278 write 0xac65400 thru 0xac71d77 (0xc978 bytes) 2024-09-14T07:10:02.410 INFO:teuthology.orchestra.run.smithi145.stdout:4279 write 0x3e7d800 thru 0x3e852d2 (0x7ad3 bytes) 2024-09-14T07:10:02.411 INFO:teuthology.orchestra.run.smithi145.stdout:4282 punch from 0x8c6fc00 to 0x8c79000, (0x9400 bytes) 2024-09-14T07:10:02.411 INFO:teuthology.orchestra.run.smithi145.stdout:4285 write 0x2dc5200 thru 0x2dcfb00 (0xa901 bytes) 2024-09-14T07:10:02.411 INFO:teuthology.orchestra.run.smithi145.stdout:4287 write 0x2d6c00 thru 0x2e4bae (0xdfaf bytes) 2024-09-14T07:10:02.411 INFO:teuthology.orchestra.run.smithi145.stdout:4292 trunc from 0xced4a00 to 0x4cfe400 2024-09-14T07:10:02.411 INFO:teuthology.orchestra.run.smithi145.stdout:4293 write 0xbf60400 thru 0xbf621ed (0x1dee bytes) 2024-09-14T07:10:02.411 INFO:teuthology.orchestra.run.smithi145.stdout:4294 write 0x79f3800 thru 0x79fddb8 (0xa5b9 bytes) 2024-09-14T07:10:02.412 INFO:teuthology.orchestra.run.smithi145.stdout:4298 write 0xdbafa00 thru 0xdbbd0ee (0xd6ef bytes) 2024-09-14T07:10:02.412 INFO:teuthology.orchestra.run.smithi145.stdout:4299 write 0x849f800 thru 0x84a3d62 (0x4563 bytes) 2024-09-14T07:10:02.412 INFO:teuthology.orchestra.run.smithi145.stdout:4300 read 0x4a26a00 thru 0x4a3111a (0xa71b bytes) 2024-09-14T07:10:02.412 INFO:teuthology.orchestra.run.smithi145.stdout:4301 read 0x3d55c00 thru 0x3d63abe (0xdebf bytes) 2024-09-14T07:10:02.412 INFO:teuthology.orchestra.run.smithi145.stdout:4304 read 0x83f0e00 thru 0x83f768f (0x6890 bytes) 2024-09-14T07:10:02.412 INFO:teuthology.orchestra.run.smithi145.stdout:4306 read 0x759ee00 thru 0x75a5ef1 (0x70f2 bytes) 2024-09-14T07:10:02.413 INFO:teuthology.orchestra.run.smithi145.stdout:4307 write 0x9701c00 thru 0x970ee38 (0xd239 bytes) 2024-09-14T07:10:02.413 INFO:teuthology.orchestra.run.smithi145.stdout:4308 punch from 0xa24f800 to 0xa251200, (0x1a00 bytes) 2024-09-14T07:10:02.413 INFO:teuthology.orchestra.run.smithi145.stdout:4310 read 0x16f9000 thru 0x1702a9c (0x9a9d bytes) 2024-09-14T07:10:02.449 INFO:teuthology.orchestra.run.smithi145.stdout:4313 trunc from 0xdbbd200 to 0x1592c00 2024-09-14T07:10:02.501 INFO:teuthology.orchestra.run.smithi145.stdout:4314 trunc from 0x1592c00 to 0x7040c00 2024-09-14T07:10:02.504 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:02.503+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:02.516 INFO:teuthology.orchestra.run.smithi145.stdout:4315 read 0x4d78c00 thru 0x4d7e86a (0x5c6b bytes) 2024-09-14T07:10:02.517 INFO:teuthology.orchestra.run.smithi145.stdout:4316 write 0x6541a00 thru 0x6551016 (0xf617 bytes) 2024-09-14T07:10:02.517 INFO:teuthology.orchestra.run.smithi145.stdout:4317 punch from 0x4deaa00 to 0x4df1600, (0x6c00 bytes) 2024-09-14T07:10:02.522 INFO:teuthology.orchestra.run.smithi145.stdout:4318 trunc from 0x7040c00 to 0x4e4aa00 2024-09-14T07:10:02.560 INFO:teuthology.orchestra.run.smithi145.stdout:4323 read 0x158a800 thru 0x1598342 (0xdb43 bytes) 2024-09-14T07:10:02.560 INFO:teuthology.orchestra.run.smithi145.stdout:4324 read 0x46a3600 thru 0x46a66bc (0x30bd bytes) 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3.2508198b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.021213010000000001, 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021229499999999998, 2024-09-14T07:10:02.597 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.598 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.598 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "waiting for peered", 2024-09-14T07:10:02.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:02.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:02.599 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:02.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.599 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:02.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:02.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:02.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:02.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:02.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:10:02.605 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:02.607 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:02.603+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:02.611 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:02.708 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:02.707+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:02.727 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:02.727 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:02.741 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:02.792 INFO:teuthology.orchestra.run.smithi145.stdout:4325 write 0x5f96e00 thru 0x5f97260 (0x461 bytes) 2024-09-14T07:10:02.808 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:02.807+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:02.812 INFO:teuthology.orchestra.run.smithi145.stdout:4327 trunc from 0x5f97400 to 0xe6dc00 2024-09-14T07:10:02.875 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:02.875 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:02.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:02.875 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:02.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.875 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:02.875 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:02.876 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 105.575976223, 2024-09-14T07:10:02.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:02.876 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.876 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.876 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:02.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:02.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:02.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:02.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:02.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 92.716500636000006, 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.886 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 92.716398350999995, 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:02.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:02.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 92.017346509999996, 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:02.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:02.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:02.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.897 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532425318000001, 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:10:02.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:10:02.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532390098, 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:10:02.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:10:02.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532334949000003, 2024-09-14T07:10:02.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:02.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532300880000001, 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:02.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:02.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532257534000003, 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:02.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532224010999997, 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532176761000002, 2024-09-14T07:10:02.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.532062513, 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531952818000001, 2024-09-14T07:10:02.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:02.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531886675000003, 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:02.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:02.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:02.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:10:02.939 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531770170999998, 2024-09-14T07:10:02.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:10:02.939 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:10:02.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:10:02.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:10:02.943 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531704515000001, 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531645195000003, 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.947 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:10:02.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:10:02.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:10:02.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.951 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531504327, 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.952 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:10:02.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:10:02.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531089219999998, 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.956 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.959 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1719 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.531021481000003, 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092498045000000001, 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1719 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.960 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341824+0000", 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:02.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341825+0000", 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.424e-06 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341831+0000", 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2630000000000003e-06 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341851+0000", 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0440000000000001e-05 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341866+0000", 2024-09-14T07:10:02.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4799e-05 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434313+0000", 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092447197999999994 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434322+0000", 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9209999999999998e-06 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:02.963 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:02.965 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:02.965 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:02.963+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:03.066 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:03.063+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:03.141 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:03.141 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:03.141 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:03.141 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:03.155 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:03.204 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:03.203+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:03.267 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:03.267+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:03.275 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:03.275 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:03.275 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:03.275 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:03.276 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:03.289 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:03.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:03.380 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:03.379+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:03.459 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:03.459 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.626522166, 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:03.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.62574781800001, 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:03.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:03.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:03.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:03.471 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 101.61456482, 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:03.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:03.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 101.31763661399999, 2024-09-14T07:10:03.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:03.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.082732790000001, 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.483 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.487 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.082170998999999, 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:03.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:03.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.081676432000002, 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.492 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:03.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:03.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:03.496 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.081059013000001, 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.080630753999998, 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.501 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:03.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.505 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.08023489, 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:03.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.510 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.079643214000001, 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:03.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.076314099999998, 2024-09-14T07:10:03.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:10:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:10:03.519 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.076273049000001, 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.520 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.076220788999997, 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.524 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:10:03.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:10:03.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:10:03.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.076181902000002, 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:10:03.528 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.532 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.076122558999998, 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:10:03.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.076068470999999, 2024-09-14T07:10:03.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:03.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.076029118000001, 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:03.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:03.546 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.075991338000001, 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:03.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:03.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.075954635000002, 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:03.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:03.555 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:03.556 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:03.557 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:03.558 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:03.558 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:03.558 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:03.555+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:03.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:03.569 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:03.569 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:03.593 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:03.593 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:03.612 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:03.634 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:03.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:10:03.683 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:03.683 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:03.683 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:03.683+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:03.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:03.787 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:03.787+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:03.827 INFO:teuthology.orchestra.run.smithi145.stdout:4330 read 0x819e00 thru 0x81f1b9 (0x53ba bytes) 2024-09-14T07:10:03.860 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:03.859+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:03.872 INFO:teuthology.orchestra.run.smithi145.stdout:4331 write 0x59d1000 thru 0x59d11d4 (0x1d5 bytes) 2024-09-14T07:10:03.872 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:03.872 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:03.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:03.918 INFO:teuthology.orchestra.run.smithi145.stdout:4333 write 0xb02b400 thru 0xb02f7ae (0x43af bytes) 2024-09-14T07:10:03.920 INFO:teuthology.orchestra.run.smithi145.stdout:4334 punch from 0xc20000 to 0xc2e000, (0xe000 bytes) 2024-09-14T07:10:03.921 INFO:teuthology.orchestra.run.smithi145.stdout:4335 write 0x2e72800 thru 0x2e7d800 (0xb001 bytes) 2024-09-14T07:10:03.921 INFO:teuthology.orchestra.run.smithi145.stdout:4337 punch from 0xa865e00 to 0xa869e00, (0x4000 bytes) 2024-09-14T07:10:03.933 INFO:teuthology.orchestra.run.smithi145.stdout:4338 trunc from 0xb02f800 to 0x3ee4200 2024-09-14T07:10:03.971 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:03.971+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:03.973 INFO:teuthology.orchestra.run.smithi145.stdout:4339 read 0x290dc00 thru 0x29159a8 (0x7da9 bytes) 2024-09-14T07:10:03.975 INFO:teuthology.orchestra.run.smithi145.stdout:4340 read 0x28c5000 thru 0x28ca94a (0x594b bytes) 2024-09-14T07:10:03.976 INFO:teuthology.orchestra.run.smithi145.stdout:4343 write 0x7bda00 thru 0x7c0516 (0x2b17 bytes) 2024-09-14T07:10:03.977 INFO:teuthology.orchestra.run.smithi145.stdout:4344 punch from 0x2451c00 to 0x2461400, (0xf800 bytes) 2024-09-14T07:10:04.004 INFO:teuthology.orchestra.run.smithi145.stdout:4346 trunc from 0x3ee4200 to 0xed31600 2024-09-14T07:10:04.026 INFO:teuthology.orchestra.run.smithi145.stdout:4348 punch from 0x9521a00 to 0x952a800, (0x8e00 bytes) 2024-09-14T07:10:04.028 INFO:teuthology.orchestra.run.smithi145.stdout:4349 read 0x8ddda00 thru 0x8de184e (0x3e4f bytes) 2024-09-14T07:10:04.028 INFO:teuthology.orchestra.run.smithi145.stdout:4350 write 0x9b32e00 thru 0x9b37454 (0x4655 bytes) 2024-09-14T07:10:04.028 INFO:teuthology.orchestra.run.smithi145.stdout:4351 punch from 0x604dc00 to 0x6053800, (0x5c00 bytes) 2024-09-14T07:10:04.028 INFO:teuthology.orchestra.run.smithi145.stdout:4353 write 0x1681400 thru 0x1683ca3 (0x28a4 bytes) 2024-09-14T07:10:04.030 INFO:teuthology.orchestra.run.smithi145.stdout:4355 read 0x8de2000 thru 0x8debe63 (0x9e64 bytes) 2024-09-14T07:10:04.030 INFO:teuthology.orchestra.run.smithi145.stdout:4356 read 0xe516c00 thru 0xe51b597 (0x4998 bytes) 2024-09-14T07:10:04.030 INFO:teuthology.orchestra.run.smithi145.stdout:4357 punch from 0x8362000 to 0x836b800, (0x9800 bytes) 2024-09-14T07:10:04.030 INFO:teuthology.orchestra.run.smithi145.stdout:4358 punch from 0x15e0000 to 0x15e4400, (0x4400 bytes) 2024-09-14T07:10:04.030 INFO:teuthology.orchestra.run.smithi145.stdout:4359 read 0x2dd2200 thru 0x2ddc67e (0xa47f bytes) 2024-09-14T07:10:04.032 INFO:teuthology.orchestra.run.smithi145.stdout:4360 flatten 2024-09-14T07:10:04.053 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:04.053 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:04.061 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:04.059+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:04.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:04.069 INFO:teuthology.orchestra.run.smithi145.stdout:4361 write 0x3891000 thru 0x389d457 (0xc458 bytes) 2024-09-14T07:10:04.071 INFO:teuthology.orchestra.run.smithi145.stdout:4365 write 0xc4fe400 thru 0xc507d95 (0x9996 bytes) 2024-09-14T07:10:04.071 INFO:teuthology.orchestra.run.smithi145.stdout:4367 write 0x3e15600 thru 0x3e15d4d (0x74e bytes) 2024-09-14T07:10:04.072 INFO:teuthology.orchestra.run.smithi145.stdout:4368 read 0x31bc200 thru 0x31c3638 (0x7439 bytes) 2024-09-14T07:10:04.084 INFO:teuthology.orchestra.run.smithi145.stdout:4369 trunc from 0xed31600 to 0x9155800 2024-09-14T07:10:04.141 INFO:teuthology.orchestra.run.smithi145.stdout:4370 read 0x6336400 thru 0x6342047 (0xbc48 bytes) 2024-09-14T07:10:04.143 INFO:teuthology.orchestra.run.smithi145.stdout:4371 punch from 0x3506600 to 0x350ba00, (0x5400 bytes) 2024-09-14T07:10:04.162 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:04.159+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:04.172 INFO:teuthology.orchestra.run.smithi145.stdout:4372 trunc from 0x9155800 to 0x73ae00 2024-09-14T07:10:04.243 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:04.243 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:04.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:04.263 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:04.259+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:04.298 INFO:teuthology.orchestra.run.smithi145.stdout:4373 write 0x9c43000 thru 0x9c4c7f6 (0x97f7 bytes) 2024-09-14T07:10:04.300 INFO:teuthology.orchestra.run.smithi145.stdout:4374 read 0x1117c00 thru 0x1120e11 (0x9212 bytes) 2024-09-14T07:10:04.312 INFO:teuthology.orchestra.run.smithi145.stdout:4376 trunc from 0x9c4c800 to 0x56e0200 2024-09-14T07:10:04.354 INFO:teuthology.orchestra.run.smithi145.stdout:4377 write 0x3fa8c00 thru 0x3fb17b0 (0x8bb1 bytes) 2024-09-14T07:10:04.356 INFO:teuthology.orchestra.run.smithi145.stdout:4378 read 0x671e00 thru 0x67addb (0x8fdc bytes) 2024-09-14T07:10:04.357 INFO:teuthology.orchestra.run.smithi145.stdout:4383 punch from 0x836200 to 0x83c800, (0x6600 bytes) 2024-09-14T07:10:04.364 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:04.363+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:04.367 INFO:teuthology.orchestra.run.smithi145.stdout:4387 trunc from 0x56e0200 to 0x1d1d400 2024-09-14T07:10:04.399 INFO:teuthology.orchestra.run.smithi145.stdout:4388 punch from 0x14aec00 to 0x14bd400, (0xe800 bytes) 2024-09-14T07:10:04.400 INFO:teuthology.orchestra.run.smithi145.stdout:4389 punch from 0xac1e00 to 0xac9800, (0x7a00 bytes) 2024-09-14T07:10:04.401 INFO:teuthology.orchestra.run.smithi145.stdout:4391 read 0x265800 thru 0x274dad (0xf5ae bytes) 2024-09-14T07:10:04.402 INFO:teuthology.orchestra.run.smithi145.stdout:4392 read 0xe2e00 thru 0xe5bad (0x2dae bytes) 2024-09-14T07:10:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:04.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:04.448 INFO:teuthology.orchestra.run.smithi145.stdout:4394 write 0x9266000 thru 0x927093a (0xa93b bytes) 2024-09-14T07:10:04.450 INFO:teuthology.orchestra.run.smithi145.stdout:4395 clone 32 order 19 su 16384 sc 14 2024-09-14T07:10:04.495 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:04.495+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:04.567 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:04.563+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:04.575 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:04.575 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:04.575 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T07:10:04.575 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-09-14T07:10:04.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T07:10:04.616 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:04.617 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:04.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:04.668 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:04.667+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:04.768 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:04.767+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:04.774 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:10:04.800 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:04.800 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:04.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:04.869 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:04.867+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:04.970 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:04.967+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:04.979 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:04.979 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:04.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:05.071 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:05.071+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:05.120 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:05.120 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:05.134 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:05.180 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:05.179+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:05.267 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:05.267 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:05.275 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:05.271+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:05.281 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:05.375 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:05.371+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:05.452 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:05.452 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:05.469 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:05.512 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:05.511+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:05.581 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:05.575+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:05.641 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:05.641 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:05.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:05.657 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:05.703 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:05.703+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:05.736 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T07:10:05.780 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:05.779+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T07:10:05.788 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:05.788 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:05.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:05.815 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:05.832 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:05.833 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:05.850 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:05.850 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:05.867 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:05.888 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:05.888 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:05.889 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:05.887+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:05.910 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:05.910 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:05.975 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:05.975 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:05.975 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:05.975 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:05.975 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:05.981 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:05.979+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:05.992 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:06.082 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:06.079+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 108.865033705, 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.005558117999996, 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.005455832999999, 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 95.306403992, 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1704 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 604160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821482799999998, 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092646624999999996, 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1704 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340420+0000", 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340421+0000", 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0999999999999997e-07 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340424+0000", 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5729999999999999e-06 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340436+0000", 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340444+0000", 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432523+0000", 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092078728999999998 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.432599+0000", 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5791999999999995e-05 2024-09-14T07:10:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433067+0000", 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046809400000000001 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1705 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 626688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821447579999997, 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093019541999999997, 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1705 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340455+0000", 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340456+0000", 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6599999999999995e-07 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340477+0000", 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0616e-05 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340490+0000", 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.311e-05 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340498+0000", 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9629999999999998e-06 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433104+0000", 2024-09-14T07:10:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092606101999999996 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433181+0000", 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6993e-05 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433475+0000", 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029399199999999997 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821392431, 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821358361999998, 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821315016, 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.203 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821281493000001, 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:06.207 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821234242999999, 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.211 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:06.215 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821119994999997, 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:06.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.219 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.821010299999998, 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.223 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.820944157, 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.227 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1714 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.820827653000002, 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093082981999999995, 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1714 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341075+0000", 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341077+0000", 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5770000000000001e-06 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341083+0000", 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000002e-06 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341105+0000", 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2569999999999999e-05 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341120+0000", 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.447e-05 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434154+0000", 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093034676999999996 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434158+0000", 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7780000000000001e-06 2024-09-14T07:10:06.231 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1715 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.820761996999998, 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093041767999999997, 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1715 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341141+0000", 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341142+0000", 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.243e-06 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341148+0000", 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4500000000000003e-06 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341166+0000", 2024-09-14T07:10:06.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8830000000000001e-05 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341180+0000", 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.394e-05 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434179+0000", 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092998288999999998 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434183+0000", 2024-09-14T07:10:06.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1716 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.820702677, 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093017457999999997, 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.236 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1716 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341200+0000", 2024-09-14T07:10:06.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341201+0000", 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.113e-06 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341231+0000", 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9869000000000002e-05 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341252+0000", 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0339999999999998e-05 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341295+0000", 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3859000000000001e-05 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434210+0000", 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092914629999999998 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434218+0000", 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1717 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.820561808999997, 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.09291075, 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.240 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1717 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341341+0000", 2024-09-14T07:10:06.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341343+0000", 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341350+0000", 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9129999999999997e-06 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341717+0000", 2024-09-14T07:10:06.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00036737800000000002 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341732+0000", 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4916000000000001e-05 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434244+0000", 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092512057999999994 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434252+0000", 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7249999999999997e-06 2024-09-14T07:10:06.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1718 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.820146702000002, 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092530256000000005, 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.244 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1718 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341756+0000", 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341757+0000", 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3e-06 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341764+0000", 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0070000000000003e-06 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341780+0000", 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6807e-05 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341810+0000", 2024-09-14T07:10:06.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9362000000000001e-05 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434279+0000", 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092468907000000003 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434286+0000", 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8730000000000007e-06 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.247 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.5858683089999999, 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.252 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:06.254 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:06.254 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:06.251+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:06.355 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:06.351+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:06.433 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:06.434 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:06.434 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:06.434 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:06.448 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:06.456 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:06.455+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:06.557 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:06.555+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:06.622 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:06.623 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:06.623 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:06.623 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:06.623 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:06.637 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:06.657 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:06.655+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:06.758 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:06.755+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.939728465, 2024-09-14T07:10:06.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:06.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:06.778 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.93895411700001, 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:06.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 104.927771119, 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:06.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:06.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:06.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.789 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 104.630842913, 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:06.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:06.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.395939089000002, 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:06.795 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:06.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.799 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.395377298, 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:06.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:06.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.804 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.394882731000003, 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:06.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.394265312000002, 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:06.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.393837052999999, 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:06.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.817 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.393441189000001, 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:06.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.392849513000002, 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.822 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:06.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1883 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389520398999998, 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083570804999999998, 2024-09-14T07:10:06.826 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1883 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380745+0000", 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380746+0000", 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7400000000000002e-07 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380750+0000", 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1479999999999999e-06 2024-09-14T07:10:06.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380762+0000", 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-05 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380774+0000", 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1276e-05 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464145+0000", 2024-09-14T07:10:06.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083371413000000005 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464218+0000", 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2472999999999995e-05 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464316+0000", 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8361000000000002e-05 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1884 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 729088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389479348000002, 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083713112000000006, 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1884 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380786+0000", 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380787+0000", 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2799999999999995e-07 2024-09-14T07:10:06.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380790+0000", 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.456e-06 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380802+0000", 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1408e-05 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380826+0000", 2024-09-14T07:10:06.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.374e-05 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464343+0000", 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083516990999999999 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464413+0000", 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0287e-05 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464499+0000", 2024-09-14T07:10:06.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6502000000000006e-05 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1885 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389427087999998, 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083841785000000002, 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.835 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1885 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380838+0000", 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380839+0000", 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1800000000000005e-07 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380843+0000", 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6899999999999998e-06 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380858+0000", 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4577e-05 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380866+0000", 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4679999999999992e-06 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464530+0000", 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083663649000000007 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464604+0000", 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4271999999999996e-05 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464680+0000", 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6310999999999995e-05 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1886 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389388201000003, 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083993737999999998, 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.839 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1886 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380877+0000", 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380878+0000", 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5799999999999999e-07 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380890+0000", 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2482e-05 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380903+0000", 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2418e-05 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380912+0000", 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8200000000000003e-06 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464714+0000", 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083802109 2024-09-14T07:10:06.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464788+0000", 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3769999999999993e-05 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464871+0000", 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3480999999999996e-05 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.843 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1887 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 796672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389328857999999, 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084122976000000002, 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1887 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380937+0000", 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380938+0000", 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380955+0000", 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7363999999999998e-05 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380967+0000", 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1851999999999999e-05 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380976+0000", 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6880000000000005e-06 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464903+0000", 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083926840000000003 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.464975+0000", 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2706000000000005e-05 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465060+0000", 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4324000000000001e-05 2024-09-14T07:10:06.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.38927477, 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:06.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.852 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389235417000002, 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:06.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389197637000002, 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.857 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:06.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:06.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:06.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.389160934000003, 2024-09-14T07:10:06.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:06.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:06.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:06.865 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:06.866 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:06.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:10:06.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:06.868 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:06.867+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:06.968 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:06.967+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:06.998 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:06.999 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:06.999 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:07.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:07.068 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:07.067+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:07.169 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:07.167+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:07.180 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:07.180 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:07.180 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:07.180 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:07.180 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:07.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:07.227 INFO:teuthology.orchestra.run.smithi145.stdout:4396 trunc from 0x9270a00 to 0x1ba6a00 2024-09-14T07:10:07.284 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:07.283+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:07.363 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.07155120199999, 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.04204357099999, 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.033396212, 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 105.36730387199999, 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 105.20057464600001, 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.267573638, 2024-09-14T07:10:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.512130466000002, 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.512112543999997, 2024-09-14T07:10:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.512099775000003, 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.404 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.512089598999999, 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.512079403999998, 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.512069384, 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 50.458568270000001, 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.298260758000001, 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.298229219, 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.298197906999999, 2024-09-14T07:10:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.298157929999999, 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.289147110999998, 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.288537547000001, 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.2627957480000003, 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.456 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:07.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:07.459 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:07.455+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:07.460 INFO:teuthology.orchestra.run.smithi145.stdout:4397 write 0xdbc4800 thru 0xdbd19f6 (0xd1f7 bytes) 2024-09-14T07:10:07.460 INFO:teuthology.orchestra.run.smithi145.stdout:4399 write 0xd945c00 thru 0xd94df40 (0x8341 bytes) 2024-09-14T07:10:07.460 INFO:teuthology.orchestra.run.smithi145.stdout:4400 read 0xa074600 thru 0xa076c5a (0x265b bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4401 write 0x2b0c200 thru 0x2b164f2 (0xa2f3 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4402 read 0x3b66200 thru 0x3b69902 (0x3703 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4403 punch from 0x76f8a00 to 0x7706400, (0xda00 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4404 read 0x158da00 thru 0x15950c8 (0x76c9 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4411 read 0xa2b1e00 thru 0xa2c17f5 (0xf9f6 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4414 write 0x9158800 thru 0x9168141 (0xf942 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4416 read 0xf10800 thru 0xf14d6e (0x456f bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4417 punch from 0xa661600 to 0xa66ec00, (0xd600 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4418 write 0x8a45c00 thru 0x8a4fb25 (0x9f26 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4420 read 0xaf39c00 thru 0xaf47875 (0xdc76 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4421 read 0x3494800 thru 0x349d548 (0x8d49 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4422 punch from 0x347b000 to 0x348a000, (0xf000 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4425 read 0xb54800 thru 0xb55fb2 (0x17b3 bytes) 2024-09-14T07:10:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:4426 write 0x1fa4e00 thru 0x1fadf65 (0x9166 bytes) 2024-09-14T07:10:07.462 INFO:teuthology.orchestra.run.smithi145.stdout:4428 clone 33 order 18 su 131072 sc 8 2024-09-14T07:10:07.559 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:07.555+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:07.634 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:07.634 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:07.634 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:07.634 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:07.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:07.696 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:07.695+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:07.760 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:07.759+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:07.776 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:07.777 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:07.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:07.802 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:10:07.867 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:07.867+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:07.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 106.254448332, 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863472495000003, 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863442696999996, 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:07.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:07.949 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863410422000001, 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:07.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863352319000001, 2024-09-14T07:10:07.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:07.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.958 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863311416999998, 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:07.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:07.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863249609, 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863215293000003, 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:07.966 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863196621999997, 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:07.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863172585999997, 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:07.974 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:07.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:07.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863152552000003, 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:07.978 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:07.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:07.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863137717000001, 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:07.982 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:07.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863123090999999, 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863108590000003, 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:07.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863090835000001, 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:07.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863075940999998, 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:07.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:07.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:08.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:08.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863061829999999, 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:08.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:08.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:08.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863039901, 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:08.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:08.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.863018590000003, 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:08.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:08.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.575667688999999, 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.015 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:08.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:08.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:08.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:08.019 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:08.020 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:08.021 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone32 from 0xdbd1a00 (overlap 0x1ba6a00) to 0x0 2024-09-14T07:10:08.021 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:08.023 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:08.019+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:08.109 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:08.122 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:08.119+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:08.127 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:08.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:08.146 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:08.163 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:08.163 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:08.183 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:08.194 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:08.194 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:08.206 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:08.206 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:08.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:08.224 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:08.223+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:08.324 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:08.323+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:08.397 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:08.398 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:08.413 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:08.463 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:08.463+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:08.526 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:08.523+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:08.534 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:08.534 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:08.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:08.590 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T07:10:08.626 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:08.623+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:08.683 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:08.683 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:08.700 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:08.728 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:08.727+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:08.829 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:08.827+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:08.879 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:08.879 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:08.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:08.930 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:08.927+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:09.031 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:09.027+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:09.075 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:09.075 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:09.090 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:09.139 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:09.139+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:09.224 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:09.224 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:09.233 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:09.231+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:09.239 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:09.334 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:09.331+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:09.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:09.464 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:09.463+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:09.477 INFO:teuthology.orchestra.run.smithi145.stdout:4429 read 0x3670800 thru 0x367b38a (0xab8b bytes) 2024-09-14T07:10:09.479 INFO:teuthology.orchestra.run.smithi145.stdout:4434 read 0x3ef5e00 thru 0x3ef6408 (0x609 bytes) 2024-09-14T07:10:09.480 INFO:teuthology.orchestra.run.smithi145.stdout:4435 write 0x4531c00 thru 0x453f8d3 (0xdcd4 bytes) 2024-09-14T07:10:09.481 INFO:teuthology.orchestra.run.smithi145.stdout:4436 read 0xc9d000 thru 0xc9e76f (0x1770 bytes) 2024-09-14T07:10:09.482 INFO:teuthology.orchestra.run.smithi145.stdout:4438 read 0xae4a400 thru 0xae53a06 (0x9607 bytes) 2024-09-14T07:10:09.483 INFO:teuthology.orchestra.run.smithi145.stdout:4439 read 0xad34200 thru 0xad3ab91 (0x6992 bytes) 2024-09-14T07:10:09.504 INFO:teuthology.orchestra.run.smithi145.stdout:4440 trunc from 0xdbd1a00 to 0x9f8d800 2024-09-14T07:10:09.536 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:09.535+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:09.544 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:09.544 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:09.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:09.563 INFO:teuthology.orchestra.run.smithi145.stdout:4443 read 0x6771000 thru 0x6774782 (0x3783 bytes) 2024-09-14T07:10:09.565 INFO:teuthology.orchestra.run.smithi145.stdout:4445 write 0x17d3800 thru 0x17d536a (0x1b6b bytes) 2024-09-14T07:10:09.599 INFO:teuthology.orchestra.run.smithi145.stdout:4446 trunc from 0x9f8d800 to 0x654c200 2024-09-14T07:10:09.636 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:09.635+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:09.660 INFO:teuthology.orchestra.run.smithi145.stdout:4447 trunc from 0x654c200 to 0x9893c00 2024-09-14T07:10:09.682 INFO:teuthology.orchestra.run.smithi145.stdout:4448 write 0x65ac400 thru 0x65b11dc (0x4ddd bytes) 2024-09-14T07:10:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:4450 write 0x588f800 thru 0x58944f3 (0x4cf4 bytes) 2024-09-14T07:10:09.684 INFO:teuthology.orchestra.run.smithi145.stdout:4455 write 0x4e61200 thru 0x4e6ff86 (0xed87 bytes) 2024-09-14T07:10:09.726 INFO:teuthology.orchestra.run.smithi145.stdout:4456 trunc from 0x9893c00 to 0x3fc5800 2024-09-14T07:10:09.737 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:09.735+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:09.738 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:09.738 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:09.753 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:09.821 INFO:teuthology.orchestra.run.smithi145.stdout:4457 write 0x956b800 thru 0x956dbd8 (0x23d9 bytes) 2024-09-14T07:10:09.822 INFO:teuthology.orchestra.run.smithi145.stdout:4459 write 0x7e1f000 thru 0x7e2e321 (0xf322 bytes) 2024-09-14T07:10:09.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:10:09.840 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:09.839+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:09.860 INFO:teuthology.orchestra.run.smithi145.stdout:4461 write 0x9ed7000 thru 0x9ee06a2 (0x96a3 bytes) 2024-09-14T07:10:09.890 INFO:teuthology.orchestra.run.smithi145.stdout:4462 write 0xbb96000 thru 0xbba3554 (0xd555 bytes) 2024-09-14T07:10:09.893 INFO:teuthology.orchestra.run.smithi145.stdout:4463 write 0x4a51c00 thru 0x4a596c6 (0x7ac7 bytes) 2024-09-14T07:10:09.893 INFO:teuthology.orchestra.run.smithi145.stdout:4465 write 0x225ea00 thru 0x22679ef (0x8ff0 bytes) 2024-09-14T07:10:09.894 INFO:teuthology.orchestra.run.smithi145.stdout:4467 read 0x8ebc200 thru 0x8ebe69d (0x249e bytes) 2024-09-14T07:10:09.894 INFO:teuthology.orchestra.run.smithi145.stdout:4468 punch from 0x3a98800 to 0x3aa7400, (0xec00 bytes) 2024-09-14T07:10:09.895 INFO:teuthology.orchestra.run.smithi145.stdout:4469 read 0x902aa00 thru 0x903275b (0x7d5c bytes) 2024-09-14T07:10:09.899 INFO:teuthology.orchestra.run.smithi145.stdout:4470 trunc from 0xbba3600 to 0xa138400 2024-09-14T07:10:09.935 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:09.935 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:09.940 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:09.939+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:09.945 INFO:teuthology.orchestra.run.smithi145.stdout:4472 punch from 0x3273800 to 0x3279e00, (0x6600 bytes) 2024-09-14T07:10:09.949 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:09.974 INFO:teuthology.orchestra.run.smithi145.stdout:4473 write 0xc44f200 thru 0xc452bb6 (0x39b7 bytes) 2024-09-14T07:10:09.975 INFO:teuthology.orchestra.run.smithi145.stdout:4474 read 0x7dfb000 thru 0x7dfb5af (0x5b0 bytes) 2024-09-14T07:10:09.975 INFO:teuthology.orchestra.run.smithi145.stdout:4476 punch from 0x2f7ba00 to 0x2f7bc00, (0x200 bytes) 2024-09-14T07:10:09.976 INFO:teuthology.orchestra.run.smithi145.stdout:4478 punch from 0x4e00200 to 0x4e01200, (0x1000 bytes) 2024-09-14T07:10:09.976 INFO:teuthology.orchestra.run.smithi145.stdout:4479 read 0x8516600 thru 0x8516f7d (0x97e bytes) 2024-09-14T07:10:09.976 INFO:teuthology.orchestra.run.smithi145.stdout:4481 punch from 0xbadd200 to 0xbadd800, (0x600 bytes) 2024-09-14T07:10:09.987 INFO:teuthology.orchestra.run.smithi145.stdout:4484 trunc from 0xc452c00 to 0x5cbde00 2024-09-14T07:10:10.006 INFO:teuthology.orchestra.run.smithi145.stdout:4486 write 0x3619600 thru 0x3619c1b (0x61c bytes) 2024-09-14T07:10:10.007 INFO:teuthology.orchestra.run.smithi145.stdout:4487 punch from 0x1f03400 to 0x1f03c00, (0x800 bytes) 2024-09-14T07:10:10.008 INFO:teuthology.orchestra.run.smithi145.stdout:4488 write 0x3c4e600 thru 0x3c50f16 (0x2917 bytes) 2024-09-14T07:10:10.008 INFO:teuthology.orchestra.run.smithi145.stdout:4489 write 0x4430e00 thru 0x44334c4 (0x26c5 bytes) 2024-09-14T07:10:10.008 INFO:teuthology.orchestra.run.smithi145.stdout:4491 write 0xbca600 thru 0xbcf4b8 (0x4eb9 bytes) 2024-09-14T07:10:10.034 INFO:teuthology.orchestra.run.smithi145.stdout:4492 write 0x93dda00 thru 0x93ddd16 (0x317 bytes) 2024-09-14T07:10:10.036 INFO:teuthology.orchestra.run.smithi145.stdout:4493 read 0xed2c00 thru 0xedcead (0xa2ae bytes) 2024-09-14T07:10:10.037 INFO:teuthology.orchestra.run.smithi145.stdout:4494 read 0x82b9c00 thru 0x82be186 (0x4587 bytes) 2024-09-14T07:10:10.040 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:10.039+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:10.049 INFO:teuthology.orchestra.run.smithi145.stdout:4495 trunc from 0x93dde00 to 0x4540200 2024-09-14T07:10:10.077 INFO:teuthology.orchestra.run.smithi145.stdout:4498 punch from 0x3975600 to 0x3976200, (0xc00 bytes) 2024-09-14T07:10:10.123 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:10.123 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:10.124 INFO:teuthology.orchestra.run.smithi145.stdout:4500 write 0xe01d600 thru 0xe021056 (0x3a57 bytes) 2024-09-14T07:10:10.127 INFO:teuthology.orchestra.run.smithi145.stdout:4501 write 0x56cf800 thru 0x56d59ef (0x61f0 bytes) 2024-09-14T07:10:10.127 INFO:teuthology.orchestra.run.smithi145.stdout:4503 write 0x3ec5e00 thru 0x3ed5859 (0xfa5a bytes) 2024-09-14T07:10:10.128 INFO:teuthology.orchestra.run.smithi145.stdout:4504 read 0x6add800 thru 0x6aea8af (0xd0b0 bytes) 2024-09-14T07:10:10.129 INFO:teuthology.orchestra.run.smithi145.stdout:4505 write 0xd30de00 thru 0xd30f5b3 (0x17b4 bytes) 2024-09-14T07:10:10.129 INFO:teuthology.orchestra.run.smithi145.stdout:4506 read 0xa7eca00 thru 0xa7fc93e (0xff3f bytes) 2024-09-14T07:10:10.129 INFO:teuthology.orchestra.run.smithi145.stdout:4509 read 0x306b800 thru 0x307ad0f (0xf510 bytes) 2024-09-14T07:10:10.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:10.142 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:10.139+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:10.162 INFO:teuthology.orchestra.run.smithi145.stdout:4511 trunc from 0xe021200 to 0x136fc00 2024-09-14T07:10:10.212 INFO:teuthology.orchestra.run.smithi145.stdout:4512 read 0xe35800 thru 0xe3ff2a (0xa72b bytes) 2024-09-14T07:10:10.217 INFO:teuthology.orchestra.run.smithi145.stdout:4515 read 0x9d8a00 thru 0x9d9fab (0x15ac bytes) 2024-09-14T07:10:10.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:10.261 INFO:teuthology.orchestra.run.smithi145.stdout:4519 write 0xe54fe00 thru 0xe554b2f (0x4d30 bytes) 2024-09-14T07:10:10.264 INFO:teuthology.orchestra.run.smithi145.stdout:4521 punch from 0x95eb600 to 0x95f5c00, (0xa600 bytes) 2024-09-14T07:10:10.275 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:10.271+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:10.280 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:10.280 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:10.293 INFO:teuthology.orchestra.run.smithi145.stdout:4523 trunc from 0xe554c00 to 0x1e63c00 2024-09-14T07:10:10.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:10.350 INFO:teuthology.orchestra.run.smithi145.stdout:4526 write 0x48a3400 thru 0x48aeeb0 (0xbab1 bytes) 2024-09-14T07:10:10.384 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:10.383+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:10.384 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:10.384 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:10.390 INFO:teuthology.orchestra.run.smithi145.stdout:4528 write 0xbc22800 thru 0xbc26bff (0x4400 bytes) 2024-09-14T07:10:10.390 INFO:teuthology.orchestra.run.smithi145.stdout:4530 read 0x8126c00 thru 0x8127cc4 (0x10c5 bytes) 2024-09-14T07:10:10.391 INFO:teuthology.orchestra.run.smithi145.stdout:4534 punch from 0x527da00 to 0x5285600, (0x7c00 bytes) 2024-09-14T07:10:10.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:10.411 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:10.414 INFO:teuthology.orchestra.run.smithi145.stdout:4535 trunc from 0xbc26c00 to 0x2a6cc00 2024-09-14T07:10:10.434 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:10.434 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:10.444 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:10.443+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:10.456 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:10.457 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:10.457 INFO:teuthology.orchestra.run.smithi145.stdout:4536 punch from 0x99ba00 to 0x9a4800, (0x8e00 bytes) 2024-09-14T07:10:10.460 INFO:teuthology.orchestra.run.smithi145.stdout:4539 punch from 0x4f9000 to 0x503e00, (0xae00 bytes) 2024-09-14T07:10:10.478 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:10.480 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:10.481 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:10.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:10.510 INFO:teuthology.orchestra.run.smithi145.stdout:4540 write 0x6a44000 thru 0x6a47393 (0x3394 bytes) 2024-09-14T07:10:10.511 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:10.512 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:10.540 INFO:teuthology.orchestra.run.smithi145.stdout:4541 write 0xa2e9a00 thru 0xa2f5dcf (0xc3d0 bytes) 2024-09-14T07:10:10.541 INFO:teuthology.orchestra.run.smithi145.stdout:4543 read 0x120d600 thru 0x1213c34 (0x6635 bytes) 2024-09-14T07:10:10.545 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:10.543+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:10.564 INFO:teuthology.orchestra.run.smithi145.stdout:4544 write 0xac68000 thru 0xac6e334 (0x6335 bytes) 2024-09-14T07:10:10.565 INFO:teuthology.orchestra.run.smithi145.stdout:4545 read 0x8933e00 thru 0x8935a7a (0x1c7b bytes) 2024-09-14T07:10:10.573 INFO:teuthology.orchestra.run.smithi145.stdout:4549 trunc from 0xac6e400 to 0x8074e00 2024-09-14T07:10:10.609 INFO:teuthology.orchestra.run.smithi145.stdout:4550 write 0x7a2aa00 thru 0x7a2cce9 (0x22ea bytes) 2024-09-14T07:10:10.611 INFO:teuthology.orchestra.run.smithi145.stdout:4551 clone 34 order 20 su 524288 sc 10 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 113.34909329, 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 113.319585659, 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 113.3109383, 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 108.64484596, 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 108.478116734, 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 104.54511572600001, 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.789672553999999, 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.789654632000001, 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.789641863, 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.789631686999996, 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.789621492000002, 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.789611471999997, 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 53.736110357999998, 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.575802845999998, 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.575771307, 2024-09-14T07:10:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.575739994999999, 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.575700017999999, 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.566689198999999, 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.566079635000001, 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5403378359999991, 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:10.730 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:10.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:10.732 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:10.731+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:10:10.810 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:10.810 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:10.811 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T07:10:10.811 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.000000 2024-09-14T07:10:10.811 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0 2024-09-14T07:10:10.831 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:10.831+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:10.918 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:10.919 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:10.932 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:10.931+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:10.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:10.973 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone33 from 0x8074e00 (overlap 0x136fc00) to 0x1f1800 2024-09-14T07:10:11.035 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:11.031+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:11.105 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:11.106 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:11.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:11.171 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:11.171+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:11.236 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:11.235+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:11.253 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 109.571623124, 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180647286999999, 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180617488999999, 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180585214000004, 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180527111000004, 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180486208999994, 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180424400999996, 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180390084999999, 2024-09-14T07:10:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180371414000007, 2024-09-14T07:10:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180347377999993, 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180327344000005, 2024-09-14T07:10:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180312509000004, 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180297882999994, 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180283381999999, 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180265626999997, 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180250732999994, 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180236621999995, 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180214692999996, 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.180193381999999, 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.892842481000002, 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.341 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:11.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:11.343 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:11.339+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:11.443 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:11.443+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:11.538 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:11.544 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:11.543+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:11.644 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:11.643+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:11.708 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:11.709 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:11.709 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:11.709 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:11.709 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:11.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:11.771 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:11.771+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:11.786 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T07:10:11.815 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.2 primary-affinity to 0 (802) 2024-09-14T07:10:11.847 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:11.843+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:11.858 INFO:teuthology.orchestra.run.smithi145.stdout:4552 trunc from 0x8074e00 to 0x802a400 2024-09-14T07:10:11.907 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.604817384, 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:11.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.559827078, 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:11.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.25789462199999, 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:11.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:11.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:11.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 109.936566888, 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:11.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:11.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.934 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.934 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:11.934 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.499438488999999, 2024-09-14T07:10:11.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:11.934 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.499427831000006, 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:11.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:11.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.943 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.499412817000007, 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:11.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.499401902000002, 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:11.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:11.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.499390391999995, 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:11.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:11.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.499377848999998, 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492531252999996, 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.965 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492483472000004, 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492470005000001, 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:11.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492459033000003, 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:11.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.978 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492448271000001, 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:11.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:11.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492432217000001, 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:11.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:11.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492415645999998, 2024-09-14T07:10:11.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.991 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492403870000004, 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:11.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:11.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.492393192999998, 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:11.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:12.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.3312167680000009, 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:12.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:12.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:12.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:12.007 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:12.009 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:12.009 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:12.007+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:12.110 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:12.107+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:12.139 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:12.139 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:12.140 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:12.140 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:12.154 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:12.210 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:12.207+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:12.311 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:12.311+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:12.313 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:12.313 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:12.313 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:12.313 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:12.313 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:12.327 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:12.413 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:12.411+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:12.497 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:12.497 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 160.19560268800001, 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:12.498 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:12.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.301609503999998, 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:12.505 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:12.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:12.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:12.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:12.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:12.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.301489625000002, 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:12.516 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:12.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:12.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.523 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.284264504999996, 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.524 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:12.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:12.529 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 61.611670117999999, 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:12.530 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:12.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:12.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837628252000002, 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:10:12.537 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.541 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837596024, 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837574009000001, 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:12.551 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837553073999999, 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:12.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837532064000001, 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.556 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:12.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:12.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837498363999998, 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837477720999999, 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:12.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837232503999999, 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.570 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.574 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837200785, 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:12.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837179026000001, 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.579 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:12.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:12.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:12.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.837146947000001, 2024-09-14T07:10:12.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:12.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:12.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.835975740999999, 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.588 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:12.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.592 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.835952777999999, 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:12.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:12.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:12.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.835931488, 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:12.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:12.602 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.835911265, 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:12.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:12.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.607 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.607 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:12.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:12.609 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:12.607+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:12.609 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:12.709 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:12.707+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3.a0eeb4c0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.87524369899999999, 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.87528712799999997, 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "waiting for peered", 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:12.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:12.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:12.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:12.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:10:12.742 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:12.752 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:12.763 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:12.763 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:12.781 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:12.797 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:12.798 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:12.810 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:12.807+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:12.816 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:12.838 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:10:12.838 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:12.848 INFO:teuthology.orchestra.run.smithi145.stdout:4553 clone 35 order 18 su 4096 sc 14 2024-09-14T07:10:12.858 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:12.919 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:12.919+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:12.923 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:12.923 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:12.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:13.012 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:13.011+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:13.060 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:13.061 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:13.075 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:13.123 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:13.123+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:13.199 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:13.200 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:13.213 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:13.211+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:13.215 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:13.315 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:13.315+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:13.380 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:13.380 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:13.394 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:13.440 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:13.439+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:13.513 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:13.513 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:13.516 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:13.515+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:13.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:13.617 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:13.615+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:13.664 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:13.664 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:13.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:13.723 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:13.723+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:13.804 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:13.804 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:13.818 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:13.815+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:13.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:10:13.919 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:13.915+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:13.955 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:13.955 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:13.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:14.020 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:14.019+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:14.038 INFO:teuthology.orchestra.run.smithi145.stdout:flattening image image_client.0-clone34 2024-09-14T07:10:14.090 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:14.090 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:14.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:14.122 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:14.119+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:14.222 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:14.219+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:14.271 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:14.271 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:14.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:14.331 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:14.331+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:14.405 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:14.405 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:14.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:14.463 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:14.463+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:14.525 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:14.523+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:14.587 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:14.588 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:14.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:14.625 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:14.623+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:14.726 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:14.723+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:14.728 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:14.729 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:14.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:14.831 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:14.831+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:14.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:14.907 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T07:10:14.914 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:14.914 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:14.927 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:14.927+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:14.929 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:15.019 INFO:teuthology.orchestra.run.smithi145.stdout:4554 read 0x58b8400 thru 0x58bf49c (0x709d bytes) 2024-09-14T07:10:15.020 INFO:teuthology.orchestra.run.smithi145.stdout:4558 read 0x2aea200 thru 0x2af3566 (0x9367 bytes) 2024-09-14T07:10:15.026 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:15.027 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:15.028 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:15.027+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:15.065 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:15.065 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:15.079 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:15.090 INFO:teuthology.orchestra.run.smithi145.stdout:4561 trunc from 0x802a400 to 0xe817800 2024-09-14T07:10:15.118 INFO:teuthology.orchestra.run.smithi145.stdout:4564 trunc from 0xe817800 to 0xbace600 2024-09-14T07:10:15.128 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:15.127+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:15.136 INFO:teuthology.orchestra.run.smithi145.stdout:4565 read 0x3545000 thru 0x354f751 (0xa752 bytes) 2024-09-14T07:10:15.140 INFO:teuthology.orchestra.run.smithi145.stdout:4568 trunc from 0xbace600 to 0xcbe9000 2024-09-14T07:10:15.215 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:15.215 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:15.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:15.229 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:15.227+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:15.253 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:15.253 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:15.284 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:15.284 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:15.314 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:15.314 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:15.329 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:15.327+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:15.343 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:15.343 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:15.372 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:15.397 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:15.397 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:15.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:15.397 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:15.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.397 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:15.397 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 163.092939331, 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:15.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:15.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:15.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:15.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:15.404 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.198946147000001, 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:15.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.198826268000005, 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:15.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:15.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.181601147999999, 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.421 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:15.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 64.509006760999995, 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:15.426 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:15.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734964894999997, 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.432 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:10:15.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:10:15.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734932667000002, 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.437 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734910652000003, 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:15.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734889717000001, 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:15.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:15.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734868706999997, 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:15.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:15.455 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734835007000001, 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:15.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:15.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734814364000002, 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:15.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734569147000002, 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:15.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.468 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734537428000003, 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:15.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:15.473 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734515668999997, 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.477 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.734483589999996, 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.478 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:15.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.733312384000001, 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:15.482 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:15.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:15.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.733289421000002, 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:15.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:15.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.733268131000003, 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:15.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 62.733247908000003, 2024-09-14T07:10:15.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:15.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:15.499 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:15.500 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:15.500 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:15.503 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:15.499+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:15.503 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4569 write 0x9d4b200 thru 0x9d55410 (0xa211 bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4571 write 0xe16b800 thru 0xe17954c (0xdd4d bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4574 write 0x21c3800 thru 0x21d2e2a (0xf62b bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4575 write 0xcf52400 thru 0xcf55579 (0x317a bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4577 read 0xd42f400 thru 0xd437cb3 (0x88b4 bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4578 write 0x1da8400 thru 0x1dac6e6 (0x42e7 bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4579 read 0xd3c7e00 thru 0xd3d1e60 (0xa061 bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4580 write 0x460e400 thru 0x4616798 (0x8399 bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4582 write 0xa9f7e00 thru 0xa9ff070 (0x7271 bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4583 read 0xa10be00 thru 0xa1151fe (0x93ff bytes) 2024-09-14T07:10:15.504 INFO:teuthology.orchestra.run.smithi145.stdout:4584 punch from 0xa36e600 to 0xa37c800, (0xe200 bytes) 2024-09-14T07:10:15.505 INFO:teuthology.orchestra.run.smithi145.stdout:4585 read 0x45a6a00 thru 0x45b4d58 (0xe359 bytes) 2024-09-14T07:10:15.535 INFO:teuthology.orchestra.run.smithi145.stdout:4588 trunc from 0xe179600 to 0xadb000 2024-09-14T07:10:15.602 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:15.599+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:15.703 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:15.703+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:15.704 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:15.704 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:15.704 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:15.704 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:15.717 INFO:teuthology.orchestra.run.smithi145.stdout:4589 trunc from 0xadb000 to 0xc86dc00 2024-09-14T07:10:15.719 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:15.733 INFO:teuthology.orchestra.run.smithi145.stdout:4593 read 0x3d06a00 thru 0x3d16999 (0xff9a bytes) 2024-09-14T07:10:15.735 INFO:teuthology.orchestra.run.smithi145.stdout:4595 read 0x9e20000 thru 0x9e251f5 (0x51f6 bytes) 2024-09-14T07:10:15.736 INFO:teuthology.orchestra.run.smithi145.stdout:4596 write 0x7fcf400 thru 0x7fd811d (0x8d1e bytes) 2024-09-14T07:10:15.736 INFO:teuthology.orchestra.run.smithi145.stdout:4597 write 0x964f000 thru 0x9651b09 (0x2b0a bytes) 2024-09-14T07:10:15.736 INFO:teuthology.orchestra.run.smithi145.stdout:4603 read 0x5d77a00 thru 0x5d8438b (0xc98c bytes) 2024-09-14T07:10:15.736 INFO:teuthology.orchestra.run.smithi145.stdout:4604 read 0x2f8a600 thru 0x2f91e5a (0x785b bytes) 2024-09-14T07:10:15.736 INFO:teuthology.orchestra.run.smithi145.stdout:4605 read 0x95eb800 thru 0x95f74fa (0xbcfb bytes) 2024-09-14T07:10:15.737 INFO:teuthology.orchestra.run.smithi145.stdout:4608 write 0xa75c800 thru 0xa765252 (0x8a53 bytes) 2024-09-14T07:10:15.737 INFO:teuthology.orchestra.run.smithi145.stdout:4609 read 0x130c000 thru 0x1312913 (0x6914 bytes) 2024-09-14T07:10:15.739 INFO:teuthology.orchestra.run.smithi145.stdout:4611 trunc from 0xc86dc00 to 0xb979600 2024-09-14T07:10:15.789 INFO:teuthology.orchestra.run.smithi145.stdout:4612 write 0x129b400 thru 0x12ab3a6 (0xffa7 bytes) 2024-09-14T07:10:15.791 INFO:teuthology.orchestra.run.smithi145.stdout:4613 punch from 0x6cee800 to 0x6ceea00, (0x200 bytes) 2024-09-14T07:10:15.792 INFO:teuthology.orchestra.run.smithi145.stdout:4615 read 0xef5a00 thru 0xf02da2 (0xd3a3 bytes) 2024-09-14T07:10:15.804 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:15.803+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:15.833 INFO:teuthology.orchestra.run.smithi145.stdout:4617 write 0xd2fc000 thru 0xd2fcfe0 (0xfe1 bytes) 2024-09-14T07:10:15.834 INFO:teuthology.orchestra.run.smithi145.stdout:4618 write 0x7976c00 thru 0x7982f61 (0xc362 bytes) 2024-09-14T07:10:15.834 INFO:teuthology.orchestra.run.smithi145.stdout:4619 punch from 0x8b6d600 to 0x8b7a000, (0xca00 bytes) 2024-09-14T07:10:15.838 INFO:teuthology.orchestra.run.smithi145.stdout:4623 trunc from 0xd2fd000 to 0xdc81200 2024-09-14T07:10:15.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:10:15.878 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:15.878 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:15.878 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:15.878 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:15.878 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:15.884 INFO:teuthology.orchestra.run.smithi145.stdout:4624 read 0x6ab2200 thru 0x6ac1561 (0xf362 bytes) 2024-09-14T07:10:15.894 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:15.923 INFO:teuthology.orchestra.run.smithi145.stdout:4626 trunc from 0xdc81200 to 0x64d000 2024-09-14T07:10:15.940 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:15.939+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:16.006 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:16.003+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:16.040 INFO:teuthology.orchestra.run.smithi145.stdout:4627 read 0x24ae00 thru 0x2557ed (0xa9ee bytes) 2024-09-14T07:10:16.046 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:16.046 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:16.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:16.046 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:16.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 118.748192527, 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.047 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:16.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 105.88871693999999, 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.053 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:16.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:16.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:16.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 105.888614655, 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.058 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:16.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 105.189562814, 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:16.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704551253000005, 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.068 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.072 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704517183999997, 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:16.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:16.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704473837999998, 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:16.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:16.080 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704440314999999, 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.081 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704393065000005, 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.085 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704278817000002, 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.089 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704169121999996, 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.704102978999998, 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:16.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:16.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 13.469027131000001, 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:16.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:16.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:16.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:16.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:16.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:16.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:16.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:16.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:16.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:16.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:16.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:16.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.1823136950000004, 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:10:16.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:10:16.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:10:16.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:10:16.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:10:16.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:10:16.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.118 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.118 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:16.118 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.56629865300000004, 2024-09-14T07:10:16.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:10:16.118 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.56623495400000001, 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:10:16.124 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.125 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:16.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:10:16.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:10:16.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.56619162000000001, 2024-09-14T07:10:16.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:10:16.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:10:16.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:10:16.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.137 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.56613474100000005, 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:10:16.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:10:16.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.56606824600000005, 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:10:16.144 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:16.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:10:16.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.56318390900000004, 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.150 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.156 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.156 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:16.158 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4628 write 0x14e7000 thru 0x14f19ea (0xa9eb bytes) 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4630 punch from 0x470c00 to 0x477c00, (0x7000 bytes) 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4631 trunc from 0x14f1a00 to 0x3a8a600 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4632 punch from 0x757e00 to 0x75f000, (0x7200 bytes) 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4633 write 0xc2b0200 thru 0xc2bd982 (0xd783 bytes) 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4634 read 0x62de00 thru 0x638e4a (0xb04b bytes) 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4635 read 0xb8da800 thru 0xb8e7f4c (0xd74d bytes) 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4637 write 0x7719400 thru 0x7723ed7 (0xaad8 bytes) 2024-09-14T07:10:16.159 INFO:teuthology.orchestra.run.smithi145.stdout:4638 trunc from 0xc2bda00 to 0xb0eb000 2024-09-14T07:10:16.159 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:16.155+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:16.171 INFO:teuthology.orchestra.run.smithi145.stdout:4639 punch from 0x714e800 to 0x715c200, (0xda00 bytes) 2024-09-14T07:10:16.173 INFO:teuthology.orchestra.run.smithi145.stdout:4640 punch from 0x3572000 to 0x3576e00, (0x4e00 bytes) 2024-09-14T07:10:16.173 INFO:teuthology.orchestra.run.smithi145.stdout:4641 read 0x2045e00 thru 0x204c839 (0x6a3a bytes) 2024-09-14T07:10:16.179 INFO:teuthology.orchestra.run.smithi145.stdout:4642 trunc from 0xb0eb000 to 0x926b000 2024-09-14T07:10:16.231 INFO:teuthology.orchestra.run.smithi145.stdout:4645 write 0xa75d800 thru 0xa75d8da (0xdb bytes) 2024-09-14T07:10:16.235 INFO:teuthology.orchestra.run.smithi145.stdout:4646 trunc from 0xa75da00 to 0xbb6e800 2024-09-14T07:10:16.258 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:16.255+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:16.272 INFO:teuthology.orchestra.run.smithi145.stdout:4653 write 0xca2a400 thru 0xca2f797 (0x5398 bytes) 2024-09-14T07:10:16.279 INFO:teuthology.orchestra.run.smithi145.stdout:4655 trunc from 0xca2f800 to 0x94aa800 2024-09-14T07:10:16.330 INFO:teuthology.orchestra.run.smithi145.stdout:4656 write 0xa337400 thru 0xa33e6ee (0x72ef bytes) 2024-09-14T07:10:16.331 INFO:teuthology.orchestra.run.smithi145.stdout:4657 write 0x4813000 thru 0x481431f (0x1320 bytes) 2024-09-14T07:10:16.331 INFO:teuthology.orchestra.run.smithi145.stdout:4659 punch from 0x2609a00 to 0x260a800, (0xe00 bytes) 2024-09-14T07:10:16.343 INFO:teuthology.orchestra.run.smithi145.stdout:4661 trunc from 0xa33e800 to 0x448bc00 2024-09-14T07:10:16.355 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:16.355 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:16.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.355 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4155 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.355 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.351991+0000", 2024-09-14T07:10:16.355 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0027698570000000001, 2024-09-14T07:10:16.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002787884, 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4155 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.351991+0000", 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.351991+0000", 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.351991+0000", 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1300000000000001e-07 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.351992+0000", 2024-09-14T07:10:16.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3829999999999999e-06 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.351995+0000", 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1580000000000001e-06 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352008+0000", 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2081e-05 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.354775+0000", 2024-09-14T07:10:16.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002767823 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4156 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352015+0000", 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0027452850000000001, 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028133199999999998, 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.359 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4156 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352015+0000", 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352015+0000", 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352015+0000", 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2899999999999999e-07 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352017+0000", 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5430000000000001e-06 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352021+0000", 2024-09-14T07:10:16.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7890000000000001e-06 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352024+0000", 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9890000000000001e-06 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.362 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4157 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352028+0000", 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0027326529999999998, 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028275510000000002, 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4157 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352028+0000", 2024-09-14T07:10:16.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352028+0000", 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352028+0000", 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4299999999999999e-07 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352029+0000", 2024-09-14T07:10:16.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1370000000000001e-06 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352032+0000", 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0580000000000002e-06 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352035+0000", 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4158 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352040+0000", 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0027207970000000001, 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028398210000000002, 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4158 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352040+0000", 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352040+0000", 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352040+0000", 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352041+0000", 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1659999999999999e-06 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352044+0000", 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1860000000000001e-06 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352047+0000", 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.773e-06 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.368 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4159 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352051+0000", 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002709763, 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028532280000000002, 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4159 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352051+0000", 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352051+0000", 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352051+0000", 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5699999999999999e-07 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352052+0000", 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2619999999999999e-06 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352056+0000", 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409999999999997e-06 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352059+0000", 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5919999999999999e-06 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4160 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352063+0000", 2024-09-14T07:10:16.372 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026978420000000002, 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028691749999999999, 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4160 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352063+0000", 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352063+0000", 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352063+0000", 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352064+0000", 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352067+0000", 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.14e-06 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352070+0000", 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0529999999999998e-06 2024-09-14T07:10:16.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4161 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352074+0000", 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026867359999999999, 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028824710000000002, 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4161 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352074+0000", 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352074+0000", 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352074+0000", 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352075+0000", 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.176e-06 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352078+0000", 2024-09-14T07:10:16.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0929999999999999e-06 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352081+0000", 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.717e-06 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4162 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352084+0000", 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002676168, 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029011290000000001, 2024-09-14T07:10:16.379 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4162 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352084+0000", 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352084+0000", 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352084+0000", 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352086+0000", 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.186e-06 2024-09-14T07:10:16.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352092+0000", 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7889999999999997e-06 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352095+0000", 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.615e-06 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4163 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.382 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352099+0000", 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026615269999999999, 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029110199999999998, 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4163 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352099+0000", 2024-09-14T07:10:16.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352099+0000", 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352099+0000", 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5199999999999998e-07 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352100+0000", 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1739999999999999e-06 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352103+0000", 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2030000000000001e-06 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352106+0000", 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6520000000000002e-06 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4164 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352110+0000", 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026508429999999999, 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029269819999999998, 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4164 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352110+0000", 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352110+0000", 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352110+0000", 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.48e-07 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352111+0000", 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.277e-06 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352115+0000", 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2050000000000004e-06 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352119+0000", 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.389 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4165 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.389 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352123+0000", 2024-09-14T07:10:16.389 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026374940000000002, 2024-09-14T07:10:16.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029377299999999999, 2024-09-14T07:10:16.389 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4165 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352123+0000", 2024-09-14T07:10:16.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352123+0000", 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352123+0000", 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5199999999999998e-07 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352124+0000", 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2049999999999999e-06 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352129+0000", 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6009999999999997e-06 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352132+0000", 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7669999999999999e-06 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4166 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352136+0000", 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026248640000000002, 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002948974, 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4166 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352136+0000", 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352136+0000", 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352136+0000", 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6e-07 2024-09-14T07:10:16.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352137+0000", 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3170000000000001e-06 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352140+0000", 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.14e-06 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352143+0000", 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4167 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352146+0000", 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0026141960000000001, 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002962583, 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4167 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352146+0000", 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352146+0000", 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352146+0000", 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6199999999999999e-07 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352150+0000", 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6150000000000001e-06 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352154+0000", 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9820000000000002e-06 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352157+0000", 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8380000000000002e-06 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4168 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352161+0000", 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025997749999999999, 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002972677, 2024-09-14T07:10:16.401 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4168 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352161+0000", 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352161+0000", 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352161+0000", 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6899999999999999e-07 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352162+0000", 2024-09-14T07:10:16.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2580000000000001e-06 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352165+0000", 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1599999999999998e-06 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352168+0000", 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6240000000000002e-06 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4169 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352171+0000", 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002589049, 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002988807, 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4169 2024-09-14T07:10:16.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352171+0000", 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352171+0000", 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352172+0000", 2024-09-14T07:10:16.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4900000000000002e-07 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352173+0000", 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2759999999999999e-06 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352181+0000", 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6389999999999998e-06 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352189+0000", 2024-09-14T07:10:16.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1380000000000001e-06 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4170 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352198+0000", 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.002562496, 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029912440000000001, 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4170 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352198+0000", 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352198+0000", 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352198+0000", 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.84e-07 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352200+0000", 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4929999999999999e-06 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352204+0000", 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7870000000000001e-06 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352207+0000", 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.768e-06 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:4171 3.1c 3.7d5d421c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:16.352245+0000", 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0025152379999999999, 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029685079999999999, 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.411 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4171 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352245+0000", 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352245+0000", 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352246+0000", 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3799999999999998e-07 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352247+0000", 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8020000000000001e-06 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352252+0000", 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.758e-06 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:16.352255+0000", 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9280000000000002e-06 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 17 2024-09-14T07:10:16.414 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:16.416 INFO:teuthology.orchestra.run.smithi145.stdout:4662 punch from 0x121a800 to 0x121c600, (0x1e00 bytes) 2024-09-14T07:10:16.416 INFO:teuthology.orchestra.run.smithi145.stdout:4663 trunc from 0x448bc00 to 0xc115e00 2024-09-14T07:10:16.416 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:16.417 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:16.415+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:16.433 INFO:teuthology.orchestra.run.smithi145.stdout:4664 punch from 0x6961e00 to 0x696fa00, (0xdc00 bytes) 2024-09-14T07:10:16.434 INFO:teuthology.orchestra.run.smithi145.stdout:4665 read 0x5d64600 thru 0x5d6d294 (0x8c95 bytes) 2024-09-14T07:10:16.434 INFO:teuthology.orchestra.run.smithi145.stdout:4667 write 0xb4e00 thru 0xbfd4f (0xaf50 bytes) 2024-09-14T07:10:16.435 INFO:teuthology.orchestra.run.smithi145.stdout:4669 write 0x82afc00 thru 0x82b8d9b (0x919c bytes) 2024-09-14T07:10:16.516 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:16.515+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:16.554 INFO:teuthology.orchestra.run.smithi145.stdout:4671 write 0xd654c00 thru 0xd65a4ef (0x58f0 bytes) 2024-09-14T07:10:16.562 INFO:teuthology.orchestra.run.smithi145.stdout:4672 trunc from 0xd65a600 to 0xa58fe00 2024-09-14T07:10:16.597 INFO:teuthology.orchestra.run.smithi145.stdout:4673 write 0x34ad800 thru 0x34b3ee9 (0x66ea bytes) 2024-09-14T07:10:16.609 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:16.609 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:16.617 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:16.615+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:16.618 INFO:teuthology.orchestra.run.smithi145.stdout:4674 trunc from 0xa58fe00 to 0xcfbe00 2024-09-14T07:10:16.624 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:16.710 INFO:teuthology.orchestra.run.smithi145.stdout:4676 trunc from 0xcfbe00 to 0xece3a00 2024-09-14T07:10:16.717 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:16.715+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:16.723 INFO:teuthology.orchestra.run.smithi145.stdout:4680 write 0x3a4ee00 thru 0x3a58c6e (0x9e6f bytes) 2024-09-14T07:10:16.724 INFO:teuthology.orchestra.run.smithi145.stdout:4681 write 0xd987600 thru 0xd98949b (0x1e9c bytes) 2024-09-14T07:10:16.727 INFO:teuthology.orchestra.run.smithi145.stdout:4682 trunc from 0xece3a00 to 0xd970a00 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.93033356199999, 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.763 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.92955921399999, 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.768 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:16.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:16.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 114.918376216, 2024-09-14T07:10:16.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:16.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:16.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 114.62144800999999, 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:16.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.784 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.386544185999995, 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.785 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:16.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:16.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.385982394999999, 2024-09-14T07:10:16.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:16.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:16.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.385487827999995, 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.798 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.384870409000001, 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:16.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:16.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:16.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.802 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.384442149999998, 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:16.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:16.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.806 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.384046286, 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.810 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.383454610000001, 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:16.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:16.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.379879867, 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:16.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.379840513999994, 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.379802733999995, 2024-09-14T07:10:16.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.379766031000003, 2024-09-14T07:10:16.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:16.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.831 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7903940970000001, 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.835 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7857216650000005, 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:10:16.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:10:16.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:10:16.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.839 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7856391729999999, 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:16.840 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7854780520000002, 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.844 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:16.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:16.848 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.785301144, 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:16.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:10:16.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:10:16.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:16.852 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:16.853 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:16.853 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:16.853 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T07:10:16.853 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:10:16.853 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:10:16.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T07:10:16.854 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:10:16.856 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:16.851+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:16.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4683 read 0x46fe400 thru 0x470d133 (0xed34 bytes) 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4684 read 0xbb91e00 thru 0xbba0c05 (0xee06 bytes) 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4686 write 0x41b3400 thru 0x41bb2e1 (0x7ee2 bytes) 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4687 write 0x6faee00 thru 0x6fb3b65 (0x4d66 bytes) 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4688 write 0x3c01c00 thru 0x3c1082a (0xec2b bytes) 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4690 trunc from 0xd970a00 to 0x6dab400 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4695 write 0x48ade00 thru 0x48b80d2 (0xa2d3 bytes) 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4696 write 0x23d3600 thru 0x23d5a73 (0x2474 bytes) 2024-09-14T07:10:16.857 INFO:teuthology.orchestra.run.smithi145.stdout:4697 read 0x31e4200 thru 0x31e768f (0x3490 bytes) 2024-09-14T07:10:16.858 INFO:teuthology.orchestra.run.smithi145.stdout:4698 read 0x1e9cc00 thru 0x1ea35fd (0x69fe bytes) 2024-09-14T07:10:16.858 INFO:teuthology.orchestra.run.smithi145.stdout:4699 trunc from 0x6dab400 to 0x1f2f200 2024-09-14T07:10:16.927 INFO:teuthology.orchestra.run.smithi145.stdout:4701 read 0xde1c00 thru 0xde3794 (0x1b95 bytes) 2024-09-14T07:10:16.954 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:16.951+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:16.960 INFO:teuthology.orchestra.run.smithi145.stdout:4707 write 0xa46ec00 thru 0xa46f0ad (0x4ae bytes) 2024-09-14T07:10:16.990 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "" 2024-09-14T07:10:16.990 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:16.991 INFO:teuthology.orchestra.run.smithi145.stdout:4709 write 0xe3b0600 thru 0xe3b3c9e (0x369f bytes) 2024-09-14T07:10:16.993 INFO:teuthology.orchestra.run.smithi145.stdout:4710 write 0x49ae200 thru 0x49bd190 (0xef91 bytes) 2024-09-14T07:10:16.993 INFO:teuthology.orchestra.run.smithi145.stdout:4713 write 0x54d9400 thru 0x54dd220 (0x3e21 bytes) 2024-09-14T07:10:16.994 INFO:teuthology.orchestra.run.smithi145.stdout:4714 read 0x32db200 thru 0x32e7015 (0xbe16 bytes) 2024-09-14T07:10:16.994 INFO:teuthology.orchestra.run.smithi145.stdout:4715 read 0x2b51e00 thru 0x2b60d5f (0xef60 bytes) 2024-09-14T07:10:16.994 INFO:teuthology.orchestra.run.smithi145.stdout:4716 write 0xb0d7400 thru 0xb0e3919 (0xc51a bytes) 2024-09-14T07:10:16.994 INFO:teuthology.orchestra.run.smithi145.stdout:4717 punch from 0x2f57c00 to 0x2f62a00, (0xae00 bytes) 2024-09-14T07:10:16.997 INFO:teuthology.orchestra.run.smithi145.stdout:4718 write 0x6fc3200 thru 0x6fce6b8 (0xb4b9 bytes) 2024-09-14T07:10:16.997 INFO:teuthology.orchestra.run.smithi145.stdout:4720 punch from 0x6910000 to 0x6919800, (0x9800 bytes) 2024-09-14T07:10:16.997 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:16.997 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:16.998 INFO:teuthology.orchestra.run.smithi145.stdout:4721 write 0x3a63600 thru 0x3a6f4e7 (0xbee8 bytes) 2024-09-14T07:10:16.998 INFO:teuthology.orchestra.run.smithi145.stdout:4722 read 0x6650a00 thru 0x66570fb (0x66fc bytes) 2024-09-14T07:10:16.998 INFO:teuthology.orchestra.run.smithi145.stdout:4725 read 0x5d0c200 thru 0x5d124fe (0x62ff bytes) 2024-09-14T07:10:16.999 INFO:teuthology.orchestra.run.smithi145.stdout:4726 read 0xd17dc00 thru 0xd186e88 (0x9289 bytes) 2024-09-14T07:10:16.999 INFO:teuthology.orchestra.run.smithi145.stdout:4728 read 0xdae0400 thru 0xdae97e6 (0x93e7 bytes) 2024-09-14T07:10:16.999 INFO:teuthology.orchestra.run.smithi145.stdout:4729 read 0xc8d0800 thru 0xc8dab1a (0xa31b bytes) 2024-09-14T07:10:17.010 INFO:teuthology.orchestra.run.smithi145.stdout:4730 trunc from 0xe3b3e00 to 0x8b12400 2024-09-14T07:10:17.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:17.055 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:17.051+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:17.129 INFO:teuthology.orchestra.run.smithi145.stdout:4732 trunc from 0x8b12400 to 0xb333800 2024-09-14T07:10:17.139 INFO:teuthology.orchestra.run.smithi145.stdout:4733 write 0x1436000 thru 0x143d615 (0x7616 bytes) 2024-09-14T07:10:17.141 INFO:teuthology.orchestra.run.smithi145.stdout:4735 read 0x48cac00 thru 0x48d76b0 (0xcab1 bytes) 2024-09-14T07:10:17.141 INFO:teuthology.orchestra.run.smithi145.stdout:4737 write 0x4a98200 thru 0x4a9fa30 (0x7831 bytes) 2024-09-14T07:10:17.151 INFO:teuthology.orchestra.run.smithi145.stdout:4739 trunc from 0xb333800 to 0x7fcf600 2024-09-14T07:10:17.155 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:17.155+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:17.190 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:17.190 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:17.190 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:17.190 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:17.190 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:17.199 INFO:teuthology.orchestra.run.smithi145.stdout:4740 write 0xc89a400 thru 0xc89d866 (0x3467 bytes) 2024-09-14T07:10:17.200 INFO:teuthology.orchestra.run.smithi145.stdout:4741 write 0x770e800 thru 0x77163c9 (0x7bca bytes) 2024-09-14T07:10:17.200 INFO:teuthology.orchestra.run.smithi145.stdout:4742 read 0xa101200 thru 0xa10eaf2 (0xd8f3 bytes) 2024-09-14T07:10:17.200 INFO:teuthology.orchestra.run.smithi145.stdout:4744 punch from 0x5ae600 to 0x5b4400, (0x5e00 bytes) 2024-09-14T07:10:17.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:17.210 INFO:teuthology.orchestra.run.smithi145.stdout:4745 trunc from 0xc89da00 to 0x87bac00 2024-09-14T07:10:17.234 INFO:teuthology.orchestra.run.smithi145.stdout:4747 read 0x775600 thru 0x77ac35 (0x5636 bytes) 2024-09-14T07:10:17.236 INFO:teuthology.orchestra.run.smithi145.stdout:4749 read 0x38f9e00 thru 0x38fdabb (0x3cbc bytes) 2024-09-14T07:10:17.237 INFO:teuthology.orchestra.run.smithi145.stdout:4750 read 0x6706200 thru 0x6707ea6 (0x1ca7 bytes) 2024-09-14T07:10:17.237 INFO:teuthology.orchestra.run.smithi145.stdout:4752 read 0x3675000 thru 0x367fca3 (0xaca4 bytes) 2024-09-14T07:10:17.237 INFO:teuthology.orchestra.run.smithi145.stdout:4753 write 0xab3200 thru 0xabb276 (0x8077 bytes) 2024-09-14T07:10:17.242 INFO:teuthology.orchestra.run.smithi145.stdout:4754 trunc from 0x87bac00 to 0xa0b4a00 2024-09-14T07:10:17.256 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:17.255+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:17.288 INFO:teuthology.orchestra.run.smithi145.stdout:4756 write 0xe35ca00 thru 0xe35dc69 (0x126a bytes) 2024-09-14T07:10:17.314 INFO:teuthology.orchestra.run.smithi145.stdout:4757 trunc from 0xe35de00 to 0xf82000 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 120.04705635400001, 2024-09-14T07:10:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 120.017548723, 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 120.008901364, 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.342809024, 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.176079798, 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.24307879, 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.487635617999999, 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.487617696000001, 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.487604927, 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.487594751000003, 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.487584556000002, 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.487574535999997, 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.434073421999997, 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.273765910000002, 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.273734371, 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.273703058999999, 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.273663081999999, 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:17.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.264652262999999, 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:17.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:17.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.264042699000001, 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:17.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:17.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:17.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:17.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.2383009, 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:17.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:17.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:17.428 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:17.429 INFO:teuthology.orchestra.run.smithi145.stdout:4758 read 0xf77600 thru 0xf78edf (0x18e0 bytes) 2024-09-14T07:10:17.430 INFO:teuthology.orchestra.run.smithi145.stdout:4761 write 0x67ab200 thru 0x67ace6e (0x1c6f bytes) 2024-09-14T07:10:17.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:17.431 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:17.427+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:17.447 INFO:teuthology.orchestra.run.smithi145.stdout:4762 write 0x9ef4c00 thru 0x9eff940 (0xad41 bytes) 2024-09-14T07:10:17.447 INFO:teuthology.orchestra.run.smithi145.stdout:4763 punch from 0x7ab0600 to 0x7ab6a00, (0x6400 bytes) 2024-09-14T07:10:17.452 INFO:teuthology.orchestra.run.smithi145.stdout:4765 trunc from 0x9effa00 to 0x84f8600 2024-09-14T07:10:17.499 INFO:teuthology.orchestra.run.smithi145.stdout:4766 write 0x7aabe00 thru 0x7ab317a (0x737b bytes) 2024-09-14T07:10:17.502 INFO:teuthology.orchestra.run.smithi145.stdout:4772 write 0x4629c00 thru 0x462e763 (0x4b64 bytes) 2024-09-14T07:10:17.502 INFO:teuthology.orchestra.run.smithi145.stdout:4773 flatten 2024-09-14T07:10:17.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:17.530 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:17.527+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:17.550 INFO:teuthology.orchestra.run.smithi145.stdout:4774 read 0x7f20e00 thru 0x7f2293c (0x1b3d bytes) 2024-09-14T07:10:17.563 INFO:teuthology.orchestra.run.smithi145.stdout:4775 write 0x874cc00 thru 0x87570ec (0xa4ed bytes) 2024-09-14T07:10:17.565 INFO:teuthology.orchestra.run.smithi145.stdout:4777 read 0x10eae00 thru 0x10efb32 (0x4d33 bytes) 2024-09-14T07:10:17.565 INFO:teuthology.orchestra.run.smithi145.stdout:4778 punch from 0x3b2b400 to 0x3b36e00, (0xba00 bytes) 2024-09-14T07:10:17.566 INFO:teuthology.orchestra.run.smithi145.stdout:4780 read 0x78f4c00 thru 0x78ff74b (0xab4c bytes) 2024-09-14T07:10:17.566 INFO:teuthology.orchestra.run.smithi145.stdout:4784 write 0x122e000 thru 0x1230f68 (0x2f69 bytes) 2024-09-14T07:10:17.567 INFO:teuthology.orchestra.run.smithi145.stdout:4787 punch from 0x1e7ac00 to 0x1e7b400, (0x800 bytes) 2024-09-14T07:10:17.567 INFO:teuthology.orchestra.run.smithi145.stdout:4790 read 0x157f600 thru 0x158f21a (0xfc1b bytes) 2024-09-14T07:10:17.567 INFO:teuthology.orchestra.run.smithi145.stdout:4791 write 0x2942a00 thru 0x294d0f5 (0xa6f6 bytes) 2024-09-14T07:10:17.568 INFO:teuthology.orchestra.run.smithi145.stdout:4794 punch from 0x1cf4800 to 0x1cfd200, (0x8a00 bytes) 2024-09-14T07:10:17.569 INFO:teuthology.orchestra.run.smithi145.stdout:4795 read 0x29f6000 thru 0x29fe7be (0x87bf bytes) 2024-09-14T07:10:17.574 INFO:teuthology.orchestra.run.smithi145.stdout:4796 trunc from 0x8757200 to 0x63df800 2024-09-14T07:10:17.620 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:17.620 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:17.620 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:17.620 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:17.631 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:17.631+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:17.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:17.652 INFO:teuthology.orchestra.run.smithi145.stdout:4798 read 0xf6ee00 thru 0xf77b04 (0x8d05 bytes) 2024-09-14T07:10:17.653 INFO:teuthology.orchestra.run.smithi145.stdout:4799 read 0x2c71800 thru 0x2c7c581 (0xad82 bytes) 2024-09-14T07:10:17.684 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:17.684 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:17.685 INFO:teuthology.orchestra.run.smithi145.stdout:4800 write 0xd48ec00 thru 0xd496d74 (0x8175 bytes) 2024-09-14T07:10:17.702 INFO:teuthology.orchestra.run.smithi145.stdout:4801 trunc from 0xd496e00 to 0x46c5e00 2024-09-14T07:10:17.703 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:17.721 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:17.722 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:17.732 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:17.731+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:17.740 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:17.741 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:17.758 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:17.758 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:17.760 INFO:teuthology.orchestra.run.smithi145.stdout:4802 punch from 0x15a0200 to 0x15a2400, (0x2200 bytes) 2024-09-14T07:10:17.762 INFO:teuthology.orchestra.run.smithi145.stdout:4804 read 0x173c000 thru 0x1743620 (0x7621 bytes) 2024-09-14T07:10:17.776 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:17.797 INFO:teuthology.orchestra.run.smithi145.stdout:4805 write 0xdf9bc00 thru 0xdfa96a5 (0xdaa6 bytes) 2024-09-14T07:10:17.798 INFO:teuthology.orchestra.run.smithi145.stdout:4807 read 0xc39f800 thru 0xc3ada28 (0xe229 bytes) 2024-09-14T07:10:17.806 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:17.806 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:17.822 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:17.822 INFO:teuthology.orchestra.run.smithi145.stdout:4808 trunc from 0xdfa9800 to 0x120d400 2024-09-14T07:10:17.863 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:17.863+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:17.934 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:17.931+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:17.939 INFO:teuthology.orchestra.run.smithi145.stdout:4812 write 0x4dc2200 thru 0x4dc6b16 (0x4917 bytes) 2024-09-14T07:10:17.940 INFO:teuthology.orchestra.run.smithi145.stdout:4813 punch from 0xf6b200 to 0xf71800, (0x6600 bytes) 2024-09-14T07:10:17.941 INFO:teuthology.orchestra.run.smithi145.stdout:4814 punch from 0x4bd3800 to 0x4bd3a00, (0x200 bytes) 2024-09-14T07:10:17.941 INFO:teuthology.orchestra.run.smithi145.stdout:4820 read 0x231dc00 thru 0x232d894 (0xfc95 bytes) 2024-09-14T07:10:17.957 INFO:teuthology.orchestra.run.smithi145.stdout:4821 trunc from 0x4dc6c00 to 0xb04de00 2024-09-14T07:10:17.970 INFO:teuthology.orchestra.run.smithi145.stdout:4822 read 0x836c000 thru 0x837ba32 (0xfa33 bytes) 2024-09-14T07:10:17.970 INFO:teuthology.orchestra.run.smithi145.stdout:4825 write 0x20c6a00 thru 0x20cfc68 (0x9269 bytes) 2024-09-14T07:10:17.971 INFO:teuthology.orchestra.run.smithi145.stdout:4827 read 0x3ec3e00 thru 0x3ec5866 (0x1a67 bytes) 2024-09-14T07:10:17.983 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:17.983 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:17.988 INFO:teuthology.orchestra.run.smithi145.stdout:4828 trunc from 0xb04de00 to 0x1eaee00 2024-09-14T07:10:17.998 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:18.027 INFO:teuthology.orchestra.run.smithi145.stdout:4829 read 0x17d7200 thru 0x17df822 (0x8623 bytes) 2024-09-14T07:10:18.039 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:18.039+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:18.044 INFO:teuthology.orchestra.run.smithi145.stdout:4830 write 0x3000c00 thru 0x300ea3e (0xde3f bytes) 2024-09-14T07:10:18.046 INFO:teuthology.orchestra.run.smithi145.stdout:4831 read 0x2956400 thru 0x295ce21 (0x6a22 bytes) 2024-09-14T07:10:18.046 INFO:teuthology.orchestra.run.smithi145.stdout:4832 read 0x2906200 thru 0x290975d (0x355e bytes) 2024-09-14T07:10:18.061 INFO:teuthology.orchestra.run.smithi145.stdout:4834 trunc from 0x300ec00 to 0x8b30c00 2024-09-14T07:10:18.112 INFO:teuthology.orchestra.run.smithi145.stdout:4835 write 0xae47a00 thru 0xae56c13 (0xf214 bytes) 2024-09-14T07:10:18.128 INFO:teuthology.orchestra.run.smithi145.stdout:4837 trunc from 0xae56e00 to 0x47fc000 2024-09-14T07:10:18.135 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:18.135+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:18.175 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:18.175 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:18.189 INFO:teuthology.orchestra.run.smithi145.stdout:4839 read 0x1ebca00 thru 0x1ebeecd (0x24ce bytes) 2024-09-14T07:10:18.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:18.191 INFO:teuthology.orchestra.run.smithi145.stdout:4841 read 0x13ec00 thru 0x143bfe (0x4fff bytes) 2024-09-14T07:10:18.193 INFO:teuthology.orchestra.run.smithi145.stdout:4843 punch from 0x2a07e00 to 0x2a16800, (0xea00 bytes) 2024-09-14T07:10:18.193 INFO:teuthology.orchestra.run.smithi145.stdout:4845 write 0x3462a00 thru 0x34710ae (0xe6af bytes) 2024-09-14T07:10:18.194 INFO:teuthology.orchestra.run.smithi145.stdout:4848 read 0x1e3c400 thru 0x1e4bfba (0xfbbb bytes) 2024-09-14T07:10:18.236 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:18.235+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:18.246 INFO:teuthology.orchestra.run.smithi145.stdout:4849 write 0xc9a0000 thru 0xc9a2408 (0x2409 bytes) 2024-09-14T07:10:18.248 INFO:teuthology.orchestra.run.smithi145.stdout:4850 read 0x3953400 thru 0x395f171 (0xbd72 bytes) 2024-09-14T07:10:18.248 INFO:teuthology.orchestra.run.smithi145.stdout:4852 write 0xb986c00 thru 0xb9933cd (0xc7ce bytes) 2024-09-14T07:10:18.249 INFO:teuthology.orchestra.run.smithi145.stdout:4853 read 0x32d3800 thru 0x32d6f19 (0x371a bytes) 2024-09-14T07:10:18.249 INFO:teuthology.orchestra.run.smithi145.stdout:4858 punch from 0x8ceae00 to 0x8cef400, (0x4600 bytes) 2024-09-14T07:10:18.251 INFO:teuthology.orchestra.run.smithi145.stdout:4859 read 0x27bda00 thru 0x27cbc4d (0xe24e bytes) 2024-09-14T07:10:18.254 INFO:teuthology.orchestra.run.smithi145.stdout:4863 trunc from 0xc9a2600 to 0xe15c000 2024-09-14T07:10:18.289 INFO:teuthology.orchestra.run.smithi145.stdout:4866 write 0xdea6600 thru 0xdeb5903 (0xf304 bytes) 2024-09-14T07:10:18.310 INFO:teuthology.orchestra.run.smithi145.stdout:4868 trunc from 0xe15c000 to 0x2efa800 2024-09-14T07:10:18.326 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:18.326 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:18.337 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:18.335+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:18.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:18.384 INFO:teuthology.orchestra.run.smithi145.stdout:4870 punch from 0x892400 to 0x8a1000, (0xec00 bytes) 2024-09-14T07:10:18.416 INFO:teuthology.orchestra.run.smithi145.stdout:4871 write 0x93fa800 thru 0x93fa97b (0x17c bytes) 2024-09-14T07:10:18.418 INFO:teuthology.orchestra.run.smithi145.stdout:4872 read 0x474ec00 thru 0x4752154 (0x3555 bytes) 2024-09-14T07:10:18.418 INFO:teuthology.orchestra.run.smithi145.stdout:4873 read 0x8440600 thru 0x8444d89 (0x478a bytes) 2024-09-14T07:10:18.423 INFO:teuthology.orchestra.run.smithi145.stdout:4874 trunc from 0x93faa00 to 0xb952200 2024-09-14T07:10:18.437 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:18.435+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:18.440 INFO:teuthology.orchestra.run.smithi145.stdout:4875 read 0xd90c00 thru 0xd93b06 (0x2f07 bytes) 2024-09-14T07:10:18.442 INFO:teuthology.orchestra.run.smithi145.stdout:4876 read 0x3f01400 thru 0x3f0cc90 (0xb891 bytes) 2024-09-14T07:10:18.457 INFO:teuthology.orchestra.run.smithi145.stdout:4879 write 0xd27b600 thru 0xd27d6d6 (0x20d7 bytes) 2024-09-14T07:10:18.461 INFO:teuthology.orchestra.run.smithi145.stdout:4881 trunc from 0xd27d800 to 0xeb39600 2024-09-14T07:10:18.478 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:18.478 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:18.480 INFO:teuthology.orchestra.run.smithi145.stdout:4884 read 0xb7ae200 thru 0xb7b1f07 (0x3d08 bytes) 2024-09-14T07:10:18.481 INFO:teuthology.orchestra.run.smithi145.stdout:4885 read 0x2b5d000 thru 0x2b6166e (0x466f bytes) 2024-09-14T07:10:18.482 INFO:teuthology.orchestra.run.smithi145.stdout:4886 write 0x882d600 thru 0x8836f0e (0x990f bytes) 2024-09-14T07:10:18.493 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:18.496 INFO:teuthology.orchestra.run.smithi145.stdout:4888 trunc from 0xeb39600 to 0x7d91400 2024-09-14T07:10:18.539 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:18.539+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:18.543 INFO:teuthology.orchestra.run.smithi145.stdout:4889 read 0x3d1a400 thru 0x3d27588 (0xd189 bytes) 2024-09-14T07:10:18.552 INFO:teuthology.orchestra.run.smithi145.stdout:4893 trunc from 0x7d91400 to 0x51d0a00 2024-09-14T07:10:18.568 INFO:teuthology.orchestra.run.smithi145.stdout:4894 punch from 0x4f39000 to 0x4f39800, (0x800 bytes) 2024-09-14T07:10:18.569 INFO:teuthology.orchestra.run.smithi145.stdout:4895 read 0x44cf200 thru 0x44dd5b0 (0xe3b1 bytes) 2024-09-14T07:10:18.570 INFO:teuthology.orchestra.run.smithi145.stdout:4896 read 0x12aea00 thru 0x12af072 (0x673 bytes) 2024-09-14T07:10:18.570 INFO:teuthology.orchestra.run.smithi145.stdout:4898 read 0x2987a00 thru 0x2992853 (0xae54 bytes) 2024-09-14T07:10:18.596 INFO:teuthology.orchestra.run.smithi145.stdout:4900 write 0xa538c00 thru 0xa543387 (0xa788 bytes) 2024-09-14T07:10:18.639 INFO:teuthology.orchestra.run.smithi145.stdout:4901 write 0xc385e00 thru 0xc38c2a3 (0x64a4 bytes) 2024-09-14T07:10:18.639 INFO:teuthology.orchestra.run.smithi145.stdout:4902 punch from 0x45fb000 to 0x45fb600, (0x600 bytes) 2024-09-14T07:10:18.639 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:18.639+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:18.642 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:18.642 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:18.663 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:18.668 INFO:teuthology.orchestra.run.smithi145.stdout:4903 write 0xe9dc400 thru 0xe9df5ad (0x31ae bytes) 2024-09-14T07:10:18.670 INFO:teuthology.orchestra.run.smithi145.stdout:4904 read 0x70f1e00 thru 0x70f32ff (0x1500 bytes) 2024-09-14T07:10:18.670 INFO:teuthology.orchestra.run.smithi145.stdout:4905 write 0xb361000 thru 0xb36776e (0x676f bytes) 2024-09-14T07:10:18.670 INFO:teuthology.orchestra.run.smithi145.stdout:4908 write 0x4397c00 thru 0x439c227 (0x4628 bytes) 2024-09-14T07:10:18.670 INFO:teuthology.orchestra.run.smithi145.stdout:4909 write 0xe2de800 thru 0xe2df9b8 (0x11b9 bytes) 2024-09-14T07:10:18.670 INFO:teuthology.orchestra.run.smithi145.stdout:4910 read 0xe2800 thru 0xebcdf (0x94e0 bytes) 2024-09-14T07:10:18.690 INFO:teuthology.orchestra.run.smithi145.stdout:4912 trunc from 0xe9df600 to 0x5632800 2024-09-14T07:10:18.740 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:18.739+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:18.806 INFO:teuthology.orchestra.run.smithi145.stdout:4913 trunc from 0x5632800 to 0xed4c400 2024-09-14T07:10:18.816 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:18.816 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:18.820 INFO:teuthology.orchestra.run.smithi145.stdout:4914 read 0x3974200 thru 0x39806b3 (0xc4b4 bytes) 2024-09-14T07:10:18.822 INFO:teuthology.orchestra.run.smithi145.stdout:4915 read 0x89c0600 thru 0x89c49e1 (0x43e2 bytes) 2024-09-14T07:10:18.822 INFO:teuthology.orchestra.run.smithi145.stdout:4916 read 0xb30c600 thru 0xb31ba92 (0xf493 bytes) 2024-09-14T07:10:18.822 INFO:teuthology.orchestra.run.smithi145.stdout:4917 read 0xc6c6400 thru 0xc6ccc3b (0x683c bytes) 2024-09-14T07:10:18.822 INFO:teuthology.orchestra.run.smithi145.stdout:4919 write 0x44b9c00 thru 0x44c8abe (0xeebf bytes) 2024-09-14T07:10:18.823 INFO:teuthology.orchestra.run.smithi145.stdout:4920 read 0x5dac200 thru 0x5db43e7 (0x81e8 bytes) 2024-09-14T07:10:18.832 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:18.835 INFO:teuthology.orchestra.run.smithi145.stdout:4922 trunc from 0xed4c400 to 0x749d200 2024-09-14T07:10:18.841 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:18.839+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:18.878 INFO:teuthology.orchestra.run.smithi145.stdout:4923 write 0x4dec000 thru 0x4deec26 (0x2c27 bytes) 2024-09-14T07:10:18.880 INFO:teuthology.orchestra.run.smithi145.stdout:4924 read 0x5fd3e00 thru 0x5fe05f7 (0xc7f8 bytes) 2024-09-14T07:10:18.880 INFO:teuthology.orchestra.run.smithi145.stdout:4925 write 0xa49200 thru 0xa4c7dc (0x35dd bytes) 2024-09-14T07:10:18.881 INFO:teuthology.orchestra.run.smithi145.stdout:4927 write 0x56ac00 thru 0x571826 (0x6c27 bytes) 2024-09-14T07:10:18.882 INFO:teuthology.orchestra.run.smithi145.stdout:4931 punch from 0x4c1b600 to 0x4c25600, (0xa000 bytes) 2024-09-14T07:10:18.883 INFO:teuthology.orchestra.run.smithi145.stdout:4932 clone 36 order 20 su 8192 sc 8 2024-09-14T07:10:18.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:10:18.941 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:18.939+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:19.006 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:19.007 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:19.023 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:19.042 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:19.039+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:19.143 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:19.143+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:19.191 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:19.191 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:19.205 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:19.244 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:19.243+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:19.345 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:19.343+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:19.377 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:19.377 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:19.391 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:19.446 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:19.443+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:19.527 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:19.527 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:19.541 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:19.547 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:19.543+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:19.561 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-14T07:10:19.647 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:19.647+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:19.715 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:19.715 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:19.730 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:19.775 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:19.775+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:19.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:19.849 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:19.847+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:19.858 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:19.859 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:19.873 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:19.950 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:19.947+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:19.950 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:19.951 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:19.967 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:19.968 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:10:19.986 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:19.986 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:20.007 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:20.007 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:20.026 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:20.026 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:20.047 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:20.047 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:20.051 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:20.051+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.228704591, 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:20.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:20.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:20.066 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.227930243, 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:20.067 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.071 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 118.21674724499999, 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.072 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.919819039, 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:20.077 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:20.082 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.684915215000004, 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:20.086 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.684353423999994, 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:20.090 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.683858857000004, 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:20.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:20.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:20.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.094 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.683241437999996, 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:20.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:20.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:20.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.098 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.682813178999993, 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.682417314999995, 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:20.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:20.106 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.681825638999996, 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.111 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.678250895999994, 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:20.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.115 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.678211543000003, 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.116 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.678173763000004, 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.678137059999997, 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.124 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:20.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:20.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.088765126, 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.129 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:10:20.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:10:20.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.084092694000001, 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:10:20.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.084010202, 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.137 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:10:20.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:20.141 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.083849081, 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:20.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.083672173, 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:20.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:20.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:20.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:20.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:10:20.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:10:20.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:20.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:20.150 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:20.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:20.196 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:20.195+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:20.254 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:20.251+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:20.328 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:20.329 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:20.329 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:20.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:20.355 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:20.355+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:20.380 INFO:teuthology.orchestra.run.smithi145.stdout:4934 read 0x44c6200 thru 0x44d19a8 (0xb7a9 bytes) 2024-09-14T07:10:20.383 INFO:teuthology.orchestra.run.smithi145.stdout:4935 write 0x4767e00 thru 0x476ea35 (0x6c36 bytes) 2024-09-14T07:10:20.384 INFO:teuthology.orchestra.run.smithi145.stdout:4936 write 0x1eb3c00 thru 0x1ebb102 (0x7503 bytes) 2024-09-14T07:10:20.385 INFO:teuthology.orchestra.run.smithi145.stdout:4939 punch from 0x4fa3a00 to 0x4fafc00, (0xc200 bytes) 2024-09-14T07:10:20.456 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:20.455+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:20.521 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:20.811 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:20.811 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:20.811 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:20.811 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:20.814 INFO:teuthology.orchestra.run.smithi145.stdout:4940 write 0xb2be000 thru 0xb2c1c29 (0x3c2a bytes) 2024-09-14T07:10:20.814 INFO:teuthology.orchestra.run.smithi145.stdout:4941 write 0xb58f200 thru 0xb591467 (0x2268 bytes) 2024-09-14T07:10:20.814 INFO:teuthology.orchestra.run.smithi145.stdout:4943 read 0x1d15000 thru 0x1d1b11b (0x611c bytes) 2024-09-14T07:10:20.814 INFO:teuthology.orchestra.run.smithi145.stdout:4944 write 0xee2e800 thru 0xee3c5c1 (0xddc2 bytes) 2024-09-14T07:10:20.814 INFO:teuthology.orchestra.run.smithi145.stdout:4949 read 0x8ed5800 thru 0x8ee398b (0xe18c bytes) 2024-09-14T07:10:20.814 INFO:teuthology.orchestra.run.smithi145.stdout:4950 trunc from 0xee3c600 to 0xb96e000 2024-09-14T07:10:20.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:20.815 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:20.811+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:20.823 INFO:teuthology.orchestra.run.smithi145.stdout:4951 write 0xd417200 thru 0xd42459a (0xd39b bytes) 2024-09-14T07:10:20.893 INFO:teuthology.orchestra.run.smithi145.stdout:4952 trunc from 0xd424600 to 0x326f800 2024-09-14T07:10:20.914 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:20.911+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:21.000 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 123.710302747, 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 123.680795116, 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 123.672147757, 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 119.006055417, 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 118.839326191, 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.90632518300001, 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.150882010999993, 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.150864088999995, 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.150851320000001, 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.150841143999997, 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.150830948999996, 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.150820929000005, 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.097319815000006, 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.937012303000003, 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.936980763999998, 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.936949452, 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.936909475, 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.927898655999996, 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.927289092000002, 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.901547293, 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:21.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:21.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.092 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:21.096 INFO:teuthology.orchestra.run.smithi145.stdout:4953 read 0x1c76600 thru 0x1c77f0b (0x190c bytes) 2024-09-14T07:10:21.096 INFO:teuthology.orchestra.run.smithi145.stdout:4954 punch from 0x14afa00 to 0x14bee00, (0xf400 bytes) 2024-09-14T07:10:21.096 INFO:teuthology.orchestra.run.smithi145.stdout:4955 trunc from 0x326f800 to 0x2c3c400 2024-09-14T07:10:21.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:21.097 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:21.091+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:21.157 INFO:teuthology.orchestra.run.smithi145.stdout:4959 read 0x1d4fe00 thru 0x1d5658b (0x678c bytes) 2024-09-14T07:10:21.159 INFO:teuthology.orchestra.run.smithi145.stdout:4962 read 0x17e2a00 thru 0x17ef753 (0xcd54 bytes) 2024-09-14T07:10:21.160 INFO:teuthology.orchestra.run.smithi145.stdout:4964 write 0x10d1600 thru 0x10d7cda (0x66db bytes) 2024-09-14T07:10:21.161 INFO:teuthology.orchestra.run.smithi145.stdout:4965 flatten 2024-09-14T07:10:21.197 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:21.191+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:21.282 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:21.282 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:21.282 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:21.282 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:21.294 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:21.291+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:21.301 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:21.391 INFO:teuthology.orchestra.run.smithi145.stdout:4966 read 0x1a56200 thru 0x1a5882d (0x262e bytes) 2024-09-14T07:10:21.395 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:21.395+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:21.421 INFO:teuthology.orchestra.run.smithi145.stdout:4967 trunc from 0x2c3c400 to 0xec0d000 2024-09-14T07:10:21.437 INFO:teuthology.orchestra.run.smithi145.stdout:4969 write 0xb0a600 thru 0xb15ea4 (0xb8a5 bytes) 2024-09-14T07:10:21.441 INFO:teuthology.orchestra.run.smithi145.stdout:4972 read 0xd359400 thru 0xd3622d8 (0x8ed9 bytes) 2024-09-14T07:10:21.441 INFO:teuthology.orchestra.run.smithi145.stdout:4977 read 0x9d4b000 thru 0x9d4d458 (0x2459 bytes) 2024-09-14T07:10:21.441 INFO:teuthology.orchestra.run.smithi145.stdout:4978 write 0x3891600 thru 0x38952c3 (0x3cc4 bytes) 2024-09-14T07:10:21.441 INFO:teuthology.orchestra.run.smithi145.stdout:4980 read 0x1ac00 thru 0x1d6d4 (0x2ad5 bytes) 2024-09-14T07:10:21.442 INFO:teuthology.orchestra.run.smithi145.stdout:4981 read 0x1e19c00 thru 0x1e1e342 (0x4743 bytes) 2024-09-14T07:10:21.443 INFO:teuthology.orchestra.run.smithi145.stdout:4982 read 0xc1d8400 thru 0xc1e7c64 (0xf865 bytes) 2024-09-14T07:10:21.444 INFO:teuthology.orchestra.run.smithi145.stdout:skipping zero length punch hole 2024-09-14T07:10:21.444 INFO:teuthology.orchestra.run.smithi145.stdout:4984 write 0xc34800 thru 0xc41732 (0xcf33 bytes) 2024-09-14T07:10:21.444 INFO:teuthology.orchestra.run.smithi145.stdout:4985 punch from 0x1f6e600 to 0x1f7b200, (0xcc00 bytes) 2024-09-14T07:10:21.445 INFO:teuthology.orchestra.run.smithi145.stdout:4986 punch from 0x12e6200 to 0x12f0200, (0xa000 bytes) 2024-09-14T07:10:21.446 INFO:teuthology.orchestra.run.smithi145.stdout:4987 write 0xc795400 thru 0xc7a149e (0xc09f bytes) 2024-09-14T07:10:21.446 INFO:teuthology.orchestra.run.smithi145.stdout:4988 read 0x7164a00 thru 0x7164fbf (0x5c0 bytes) 2024-09-14T07:10:21.446 INFO:teuthology.orchestra.run.smithi145.stdout:4991 punch from 0xd711400 to 0xd71a400, (0x9000 bytes) 2024-09-14T07:10:21.461 INFO:teuthology.orchestra.run.smithi145.stdout:4992 trunc from 0xec0d000 to 0x5c8c400 2024-09-14T07:10:21.480 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:21.481 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:21.496 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:21.495+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:21.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:21.502 INFO:teuthology.orchestra.run.smithi145.stdout:4994 punch from 0x5380a00 to 0x538be00, (0xb400 bytes) 2024-09-14T07:10:21.505 INFO:teuthology.orchestra.run.smithi145.stdout:4995 read 0x3565a00 thru 0x3572fbb (0xd5bc bytes) 2024-09-14T07:10:21.505 INFO:teuthology.orchestra.run.smithi145.stdout:4996 read 0x3970e00 thru 0x397ce49 (0xc04a bytes) 2024-09-14T07:10:21.506 INFO:teuthology.orchestra.run.smithi145.stdout:4998 write 0x4d67000 thru 0x4d7571b (0xe71c bytes) 2024-09-14T07:10:21.506 INFO:teuthology.orchestra.run.smithi145.stdout:4999 read 0x17d200 thru 0x18b8e6 (0xe6e7 bytes) 2024-09-14T07:10:21.544 INFO:teuthology.orchestra.run.smithi145.stdout:5000 trunc from 0x5c8c400 to 0xa9fc00 2024-09-14T07:10:21.597 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:21.595+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:21.606 INFO:teuthology.orchestra.run.smithi145.stdout:5002 read 0x45c400 thru 0x467678 (0xb279 bytes) 2024-09-14T07:10:21.624 INFO:teuthology.orchestra.run.smithi145.stdout:5003 trunc from 0xa9fc00 to 0x616d600 2024-09-14T07:10:21.637 INFO:teuthology.orchestra.run.smithi145.stdout:5004 read 0x37f3a00 thru 0x37f63f9 (0x29fa bytes) 2024-09-14T07:10:21.639 INFO:teuthology.orchestra.run.smithi145.stdout:5009 punch from 0x20d4000 to 0x20e3200, (0xf200 bytes) 2024-09-14T07:10:21.651 INFO:teuthology.orchestra.run.smithi145.stdout:5013 trunc from 0x616d600 to 0x19c6c00 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 119.997427991, 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.683 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606452153999996, 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.687 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606422355999996, 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:21.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:21.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606390081000001, 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.695 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606331978, 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:21.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:21.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:21.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606291076000005, 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.700 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:21.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.703 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606229268000007, 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.704 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:21.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:21.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606194951999996, 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.708 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606176281000003, 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:21.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606152245000004, 2024-09-14T07:10:21.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:21.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606132211000002, 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:21.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606117376, 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:21.724 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:21.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606102750000005, 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:21.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:21.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606088248999995, 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.733 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:21.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:21.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606070493999994, 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:21.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606055600000005, 2024-09-14T07:10:21.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606041489000006, 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:21.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:21.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.606019559999993, 2024-09-14T07:10:21.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.605998248999995, 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:21.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.318647347999999, 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:21.759 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:21.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:21.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:21.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.764 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:21.764 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:21.764 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:5014 write 0xde34a00 thru 0xde37f52 (0x3553 bytes) 2024-09-14T07:10:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:5019 write 0x6091200 thru 0x60a1136 (0xff37 bytes) 2024-09-14T07:10:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:5020 read 0xce01000 thru 0xce02460 (0x1461 bytes) 2024-09-14T07:10:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:5021 read 0x689d600 thru 0x689f00e (0x1a0f bytes) 2024-09-14T07:10:21.766 INFO:teuthology.orchestra.run.smithi145.stdout:5023 trunc from 0xde38000 to 0x3f26c00 2024-09-14T07:10:21.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:21.767 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:21.763+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:21.866 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:21.863+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:21.870 INFO:teuthology.orchestra.run.smithi145.stdout:5024 write 0xcbbc200 thru 0xcbc9b8f (0xd990 bytes) 2024-09-14T07:10:21.872 INFO:teuthology.orchestra.run.smithi145.stdout:5025 punch from 0x8c00800 to 0x8c0e600, (0xde00 bytes) 2024-09-14T07:10:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:5026 punch from 0x47a0200 to 0x47a1800, (0x1600 bytes) 2024-09-14T07:10:21.873 INFO:teuthology.orchestra.run.smithi145.stdout:5028 write 0x5b4400 thru 0x5c0219 (0xbe1a bytes) 2024-09-14T07:10:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:5032 read 0x5551e00 thru 0x55551e2 (0x33e3 bytes) 2024-09-14T07:10:21.875 INFO:teuthology.orchestra.run.smithi145.stdout:5033 punch from 0x5b0c000 to 0x5b17a00, (0xba00 bytes) 2024-09-14T07:10:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:5037 read 0x3d07c00 thru 0x3d1710a (0xf50b bytes) 2024-09-14T07:10:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:5038 read 0x91bd800 thru 0x91c4833 (0x7034 bytes) 2024-09-14T07:10:21.876 INFO:teuthology.orchestra.run.smithi145.stdout:5041 write 0x2032600 thru 0x203a25e (0x7c5f bytes) 2024-09-14T07:10:21.947 INFO:teuthology.orchestra.run.smithi145.stdout:5042 write 0xd8f6600 thru 0xd8fdca7 (0x76a8 bytes) 2024-09-14T07:10:21.949 INFO:teuthology.orchestra.run.smithi145.stdout:5045 punch from 0x4d8a400 to 0x4d90600, (0x6200 bytes) 2024-09-14T07:10:21.950 INFO:teuthology.orchestra.run.smithi145.stdout:5047 punch from 0xa39fa00 to 0xa3a8a00, (0x9000 bytes) 2024-09-14T07:10:21.950 INFO:teuthology.orchestra.run.smithi145.stdout:5050 read 0x5720400 thru 0x572f97d (0xf57e bytes) 2024-09-14T07:10:21.950 INFO:teuthology.orchestra.run.smithi145.stdout:5051 write 0x1efb000 thru 0x1f039e6 (0x89e7 bytes) 2024-09-14T07:10:21.951 INFO:teuthology.orchestra.run.smithi145.stdout:5053 read 0x9cecc00 thru 0x9cedf7d (0x137e bytes) 2024-09-14T07:10:21.951 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:21.951 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:21.951 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:21.951 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:21.955 INFO:teuthology.orchestra.run.smithi145.stdout:5057 trunc from 0xd8fde00 to 0xba72c00 2024-09-14T07:10:21.967 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:21.963+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:21.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:21.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:10:22.007 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:22.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T07:10:22.019 INFO:teuthology.orchestra.run.smithi145.stdout:5058 write 0xdc0c200 thru 0xdc0fb0c (0x390d bytes) 2024-09-14T07:10:22.020 INFO:teuthology.orchestra.run.smithi145.stdout:5059 write 0x1318c00 thru 0x1324cb4 (0xc0b5 bytes) 2024-09-14T07:10:22.020 INFO:teuthology.orchestra.run.smithi145.stdout:5064 punch from 0x8e14e00 to 0x8e1c800, (0x7a00 bytes) 2024-09-14T07:10:22.022 INFO:teuthology.orchestra.run.smithi145.stdout:5065 write 0xd871400 thru 0xd87f322 (0xdf23 bytes) 2024-09-14T07:10:22.022 INFO:teuthology.orchestra.run.smithi145.stdout:5066 write 0x7ed4400 thru 0x7ee29d7 (0xe5d8 bytes) 2024-09-14T07:10:22.022 INFO:teuthology.orchestra.run.smithi145.stdout:5067 write 0x66e00 thru 0x74b76 (0xdd77 bytes) 2024-09-14T07:10:22.023 INFO:teuthology.orchestra.run.smithi145.stdout:5068 read 0x9a4ca00 thru 0x9a4ed75 (0x2376 bytes) 2024-09-14T07:10:22.023 INFO:teuthology.orchestra.run.smithi145.stdout:5069 punch from 0x4130000 to 0x4130200, (0x200 bytes) 2024-09-14T07:10:22.025 INFO:teuthology.orchestra.run.smithi145.stdout:5070 punch from 0x1933c00 to 0x193c800, (0x8c00 bytes) 2024-09-14T07:10:22.025 INFO:teuthology.orchestra.run.smithi145.stdout:5071 punch from 0xd803600 to 0xd80a600, (0x7000 bytes) 2024-09-14T07:10:22.026 INFO:teuthology.orchestra.run.smithi145.stdout:5072 read 0xb985e00 thru 0xb991647 (0xb848 bytes) 2024-09-14T07:10:22.026 INFO:teuthology.orchestra.run.smithi145.stdout:5073 read 0xb94e00 thru 0xba0c3f (0xbe40 bytes) 2024-09-14T07:10:22.060 INFO:teuthology.orchestra.run.smithi145.stdout:5074 read 0xd408200 thru 0xd40c1f1 (0x3ff2 bytes) 2024-09-14T07:10:22.060 INFO:teuthology.orchestra.run.smithi145.stdout:5075 read 0x21a3000 thru 0x21b1c0e (0xec0f bytes) 2024-09-14T07:10:22.062 INFO:teuthology.orchestra.run.smithi145.stdout:5076 write 0x31d5c00 thru 0x31da44a (0x484b bytes) 2024-09-14T07:10:22.062 INFO:teuthology.orchestra.run.smithi145.stdout:5080 read 0x5936200 thru 0x59435db (0xd3dc bytes) 2024-09-14T07:10:22.063 INFO:teuthology.orchestra.run.smithi145.stdout:5084 write 0xb036600 thru 0xb03b681 (0x5082 bytes) 2024-09-14T07:10:22.064 INFO:teuthology.orchestra.run.smithi145.stdout:5087 punch from 0x464f000 to 0x4655e00, (0x6e00 bytes) 2024-09-14T07:10:22.065 INFO:teuthology.orchestra.run.smithi145.stdout:5091 write 0xdc0e00 thru 0xdc5732 (0x4933 bytes) 2024-09-14T07:10:22.065 INFO:teuthology.orchestra.run.smithi145.stdout:5094 punch from 0xe63400 to 0xe70600, (0xd200 bytes) 2024-09-14T07:10:22.066 INFO:teuthology.orchestra.run.smithi145.stdout:5095 write 0xc08f600 thru 0xc094991 (0x5392 bytes) 2024-09-14T07:10:22.066 INFO:teuthology.orchestra.run.smithi145.stdout:5096 punch from 0xbcdda00 to 0xbce7600, (0x9c00 bytes) 2024-09-14T07:10:22.067 INFO:teuthology.orchestra.run.smithi145.stdout:5097 write 0x306d200 thru 0x306e8be (0x16bf bytes) 2024-09-14T07:10:22.067 INFO:teuthology.orchestra.run.smithi145.stdout:5098 punch from 0xb9ca00 to 0xba2400, (0x5a00 bytes) 2024-09-14T07:10:22.067 INFO:teuthology.orchestra.run.smithi145.stdout:5101 read 0xbfb2200 thru 0xbfbfd20 (0xdb21 bytes) 2024-09-14T07:10:22.067 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:22.067+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:22.106 INFO:teuthology.orchestra.run.smithi145.stdout:5102 write 0xe784a00 thru 0xe78a482 (0x5a83 bytes) 2024-09-14T07:10:22.107 INFO:teuthology.orchestra.run.smithi145.stdout:5104 read 0x493bc00 thru 0x4946b13 (0xaf14 bytes) 2024-09-14T07:10:22.108 INFO:teuthology.orchestra.run.smithi145.stdout:5106 write 0x8cc1800 thru 0x8cc1ae8 (0x2e9 bytes) 2024-09-14T07:10:22.109 INFO:teuthology.orchestra.run.smithi145.stdout:5107 read 0x2753600 thru 0x2753a0b (0x40c bytes) 2024-09-14T07:10:22.109 INFO:teuthology.orchestra.run.smithi145.stdout:5111 read 0xcffba00 thru 0xcfff621 (0x3c22 bytes) 2024-09-14T07:10:22.109 INFO:teuthology.orchestra.run.smithi145.stdout:5114 write 0xb84ce00 thru 0xb84e178 (0x1379 bytes) 2024-09-14T07:10:22.121 INFO:teuthology.orchestra.run.smithi145.stdout:5116 trunc from 0xe78a600 to 0x98fb600 2024-09-14T07:10:22.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:22.153 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:22.153 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:22.153 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:22.153 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:22.153 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:22.164 INFO:teuthology.orchestra.run.smithi145.stdout:5119 read 0x8519a00 thru 0x851bcdc (0x22dd bytes) 2024-09-14T07:10:22.165 INFO:teuthology.orchestra.run.smithi145.stdout:5120 read 0x30ef400 thru 0x30f3b97 (0x4798 bytes) 2024-09-14T07:10:22.166 INFO:teuthology.orchestra.run.smithi145.stdout:5122 write 0x350c600 thru 0x351928a (0xcc8b bytes) 2024-09-14T07:10:22.166 INFO:teuthology.orchestra.run.smithi145.stdout:5125 write 0x5cd7200 thru 0x5cda66a (0x346b bytes) 2024-09-14T07:10:22.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:22.173 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:22.171+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:22.187 INFO:teuthology.orchestra.run.smithi145.stdout:5128 write 0x9f51200 thru 0x9f585d6 (0x73d7 bytes) 2024-09-14T07:10:22.188 INFO:teuthology.orchestra.run.smithi145.stdout:5130 read 0x1828400 thru 0x1830597 (0x8198 bytes) 2024-09-14T07:10:22.190 INFO:teuthology.orchestra.run.smithi145.stdout:5132 read 0x44b5800 thru 0x44bbead (0x66ae bytes) 2024-09-14T07:10:22.215 INFO:teuthology.orchestra.run.smithi145.stdout:5135 write 0xab94000 thru 0xab955d3 (0x15d4 bytes) 2024-09-14T07:10:22.215 INFO:teuthology.orchestra.run.smithi145.stdout:5138 read 0x9573200 thru 0x9575101 (0x1f02 bytes) 2024-09-14T07:10:22.216 INFO:teuthology.orchestra.run.smithi145.stdout:5140 punch from 0x7845200 to 0x784f800, (0xa600 bytes) 2024-09-14T07:10:22.217 INFO:teuthology.orchestra.run.smithi145.stdout:5143 punch from 0x1cc7c00 to 0x1ccae00, (0x3200 bytes) 2024-09-14T07:10:22.231 INFO:teuthology.orchestra.run.smithi145.stdout:5144 trunc from 0xab95600 to 0x3339400 2024-09-14T07:10:22.268 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:22.267+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:22.273 INFO:teuthology.orchestra.run.smithi145.stdout:5146 write 0xa18600 thru 0xa1f280 (0x6c81 bytes) 2024-09-14T07:10:22.274 INFO:teuthology.orchestra.run.smithi145.stdout:5148 punch from 0x2203a00 to 0x2211c00, (0xe200 bytes) 2024-09-14T07:10:22.275 INFO:teuthology.orchestra.run.smithi145.stdout:5149 read 0x12af000 thru 0x12bae0d (0xbe0e bytes) 2024-09-14T07:10:22.278 INFO:teuthology.orchestra.run.smithi145.stdout:5150 read 0x7c1200 thru 0x7c574f (0x4550 bytes) 2024-09-14T07:10:22.279 INFO:teuthology.orchestra.run.smithi145.stdout:5151 read 0x542800 thru 0x550548 (0xdd49 bytes) 2024-09-14T07:10:22.298 INFO:teuthology.orchestra.run.smithi145.stdout:5152 trunc from 0x3339400 to 0x9f13800 2024-09-14T07:10:22.310 INFO:teuthology.orchestra.run.smithi145.stdout:5153 read 0x4585e00 thru 0x45955ad (0xf7ae bytes) 2024-09-14T07:10:22.312 INFO:teuthology.orchestra.run.smithi145.stdout:5154 punch from 0x26f1600 to 0x26fe000, (0xca00 bytes) 2024-09-14T07:10:22.324 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:22.327 INFO:teuthology.orchestra.run.smithi145.stdout:5155 write 0xa384a00 thru 0xa3949a7 (0xffa8 bytes) 2024-09-14T07:10:22.329 INFO:teuthology.orchestra.run.smithi145.stdout:5157 write 0x54a0200 thru 0x54a2e09 (0x2c0a bytes) 2024-09-14T07:10:22.329 INFO:teuthology.orchestra.run.smithi145.stdout:5158 read 0x7848a00 thru 0x784984d (0xe4e bytes) 2024-09-14T07:10:22.329 INFO:teuthology.orchestra.run.smithi145.stdout:5159 read 0x65b9600 thru 0x65bf546 (0x5f47 bytes) 2024-09-14T07:10:22.330 INFO:teuthology.orchestra.run.smithi145.stdout:5160 clone 37 order 19 su 262144 sc 13 2024-09-14T07:10:22.343 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:22.343 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.04798418, 2024-09-14T07:10:22.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:22.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.002993874, 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:22.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 120.70106141799999, 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:22.364 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:22.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:22.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:22.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:22.371 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 120.379733684, 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:22.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:22.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:22.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.942605284999999, 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:22.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.942594627000005, 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:22.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.384 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:22.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:22.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.942579613000007, 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.389 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.942568698000002, 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:22.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:22.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.942557187999995, 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.942544644999998, 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:22.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935698048999996, 2024-09-14T07:10:22.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:22.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.415 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935650268000003, 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.416 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:22.420 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935636801000001, 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:22.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:22.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935625829000003, 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.426 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:22.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:22.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.430 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935615067000001, 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:22.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935599013000001, 2024-09-14T07:10:22.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:22.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:22.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.441 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935582441999998, 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:22.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:22.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935570666000004, 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:22.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:22.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:22.451 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.935559988999998, 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:22.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:22.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:22.457 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.774383564000001, 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:22.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:10:22.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:22.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:10:22.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:22.465 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:22.467 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:22.467 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:22.468 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:22.469 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:22.469 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:22.469 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:22.470 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:22.470 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:22.463+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:22.568 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:22.567+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:22.601 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:22.601 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:22.614 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:22.669 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:22.667+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:22.736 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:22.736 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:22.749 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:22.770 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:22.767+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:22.866 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:22.866 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:22.870 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:22.867+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:22.879 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:22.972 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:22.971+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T07:10:22.995 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:10:23.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T07:10:23.046 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:23.046 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:23.061 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:23.111 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:23.111+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:23.174 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:23.171+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:23.223 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:23.224 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:23.237 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:23.274 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:23.271+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:23.369 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:23.369 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:23.374 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:23.371+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:23.383 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:23.476 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:23.475+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:23.562 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:23.562 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:23.576 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:23.575+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:23.577 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:23.617 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-14T07:10:23.677 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:23.675+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:23.774 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:23.774 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:23.777 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:23.775+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:23.793 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:23.880 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:23.879+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:23.980 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:23.979+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:23.982 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:23.982 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:24.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T07:10:24.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T07:10:24.080 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:24.079+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:24.143 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:24.143 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:24.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:24.180 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:24.179+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:24.280 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:24.281 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:24.281 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:24.279+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:24.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:24.382 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:24.379+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:24.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:24.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:24.495 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:24.495+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:24.583 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:24.579+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:24.624 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:24.624 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:24.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:24.648 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:24.649 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:24.667 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:24.683 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:24.683+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:24.684 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:24.684 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:24.704 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:24.725 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:24.747 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:24.747 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:24.759 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:24.759 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:24.759 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:24.759 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:24.759 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:24.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:24.785 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:24.783+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:24.875 INFO:teuthology.orchestra.run.smithi145.stdout:5161 read 0x64fce00 thru 0x65086df (0xb8e0 bytes) 2024-09-14T07:10:24.886 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:24.883+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:24.917 INFO:teuthology.orchestra.run.smithi145.stdout:5163 write 0xbad3a00 thru 0xbad9e16 (0x6417 bytes) 2024-09-14T07:10:24.921 INFO:teuthology.orchestra.run.smithi145.stdout:5165 trunc from 0xbada000 to 0xa1c3c00 2024-09-14T07:10:24.935 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 123.25377127900001, 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.940 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862795442000007, 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:24.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.944 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862765644000007, 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:24.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:24.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862733368999997, 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:24.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862675265999997, 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.955 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862634364000002, 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:24.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:24.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862572556000003, 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:24.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862538240000006, 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.964 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:24.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862519569, 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:24.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862495533000001, 2024-09-14T07:10:24.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:24.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862475498999999, 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:24.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862460663999997, 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.982 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:24.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862446038000002, 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:24.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862431537000006, 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:24.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:24.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862413782000004, 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:24.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:24.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:24.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862398888000001, 2024-09-14T07:10:25.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:25.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.004 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862384777000003, 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:25.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862362848000004, 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:25.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.012 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.862341537000006, 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:25.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:25.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.574990635999995, 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.022 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.022 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:25.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:25.024 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T07:10:25.024 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:25.023+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:25.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:10:25.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T07:10:25.070 INFO:teuthology.orchestra.run.smithi145.stdout:5167 trunc from 0xa1c3c00 to 0x1a1c600 2024-09-14T07:10:25.124 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:25.123+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:25.126 INFO:teuthology.orchestra.run.smithi145.stdout:5172 punch from 0x5eee00 to 0x5fa600, (0xb800 bytes) 2024-09-14T07:10:25.174 INFO:teuthology.orchestra.run.smithi145.stdout:5173 write 0xa460600 thru 0xa4661fe (0x5bff bytes) 2024-09-14T07:10:25.175 INFO:teuthology.orchestra.run.smithi145.stdout:5174 write 0x354ca00 thru 0x35532f1 (0x68f2 bytes) 2024-09-14T07:10:25.175 INFO:teuthology.orchestra.run.smithi145.stdout:5181 write 0x55efa00 thru 0x55fe915 (0xef16 bytes) 2024-09-14T07:10:25.209 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5950.0:761 3.9 e140/57 3:96826862:::rbd_header.116144317fe8:head v 140'566, mlcod=140'564)", 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:25.209418+0000", 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00067621300000000005, 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069541099999999995, 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209418+0000", 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209418+0000", 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209420+0000", 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.872e-06 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209431+0000", 2024-09-14T07:10:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0722e-05 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209452+0000", 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0508000000000001e-05 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209464+0000", 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.236e-05 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209551+0000", 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6958000000000003e-05 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:25.209579+0000", 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8277e-05 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:10:25.213 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:25.217 INFO:teuthology.orchestra.run.smithi145.stdout:5183 write 0xaafb200 thru 0xab03874 (0x8675 bytes) 2024-09-14T07:10:25.218 INFO:teuthology.orchestra.run.smithi145.stdout:5184 write 0x21b7e00 thru 0x21b8024 (0x225 bytes) 2024-09-14T07:10:25.218 INFO:teuthology.orchestra.run.smithi145.stdout:5185 read 0x3c02a00 thru 0x3c069be (0x3fbf bytes) 2024-09-14T07:10:25.219 INFO:teuthology.orchestra.run.smithi145.stdout:5187 write 0x14a5c00 thru 0x14ac2e2 (0x66e3 bytes) 2024-09-14T07:10:25.225 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:25.223+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:25.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:25.258 INFO:teuthology.orchestra.run.smithi145.stdout:5188 write 0xac06800 thru 0xac11076 (0xa877 bytes) 2024-09-14T07:10:25.260 INFO:teuthology.orchestra.run.smithi145.stdout:5190 write 0x3f53a00 thru 0x3f5f59e (0xbb9f bytes) 2024-09-14T07:10:25.305 INFO:teuthology.orchestra.run.smithi145.stdout:5191 write 0xd2e2400 thru 0xd2ed704 (0xb305 bytes) 2024-09-14T07:10:25.307 INFO:teuthology.orchestra.run.smithi145.stdout:5192 read 0xc8a3400 thru 0xc8b2510 (0xf111 bytes) 2024-09-14T07:10:25.307 INFO:teuthology.orchestra.run.smithi145.stdout:5193 write 0x9f9c400 thru 0x9fa8d83 (0xc984 bytes) 2024-09-14T07:10:25.307 INFO:teuthology.orchestra.run.smithi145.stdout:5194 write 0xf80600 thru 0xf85ac6 (0x54c7 bytes) 2024-09-14T07:10:25.308 INFO:teuthology.orchestra.run.smithi145.stdout:5195 read 0xc1fd400 thru 0xc20b8f1 (0xe4f2 bytes) 2024-09-14T07:10:25.308 INFO:teuthology.orchestra.run.smithi145.stdout:5196 read 0x9764a00 thru 0x97733ce (0xe9cf bytes) 2024-09-14T07:10:25.325 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:25.323+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:25.345 INFO:teuthology.orchestra.run.smithi145.stdout:5201 write 0xdab7200 thru 0xdabe64d (0x744e bytes) 2024-09-14T07:10:25.358 INFO:teuthology.orchestra.run.smithi145.stdout:5202 trunc from 0xdabe800 to 0x7e7fe00 2024-09-14T07:10:25.390 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:25.390 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:25.397 INFO:teuthology.orchestra.run.smithi145.stdout:5203 read 0x788ec00 thru 0x789e188 (0xf589 bytes) 2024-09-14T07:10:25.400 INFO:teuthology.orchestra.run.smithi145.stdout:5205 read 0x738d000 thru 0x7394d4d (0x7d4e bytes) 2024-09-14T07:10:25.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:25.420 INFO:teuthology.orchestra.run.smithi145.stdout:5207 write 0x8895a00 thru 0x889f328 (0x9929 bytes) 2024-09-14T07:10:25.422 INFO:teuthology.orchestra.run.smithi145.stdout:5208 write 0x584c00 thru 0x588013 (0x3414 bytes) 2024-09-14T07:10:25.423 INFO:teuthology.orchestra.run.smithi145.stdout:5209 read 0x27e6800 thru 0x27f1994 (0xb195 bytes) 2024-09-14T07:10:25.424 INFO:teuthology.orchestra.run.smithi145.stdout:5210 write 0x678b800 thru 0x678ccca (0x14cb bytes) 2024-09-14T07:10:25.424 INFO:teuthology.orchestra.run.smithi145.stdout:5216 read 0x51ca800 thru 0x51cd7e6 (0x2fe7 bytes) 2024-09-14T07:10:25.447 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:25.447+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:25.478 INFO:teuthology.orchestra.run.smithi145.stdout:5217 write 0xc707c00 thru 0xc70f0c0 (0x74c1 bytes) 2024-09-14T07:10:25.480 INFO:teuthology.orchestra.run.smithi145.stdout:5218 read 0x2b28000 thru 0x2b30799 (0x879a bytes) 2024-09-14T07:10:25.480 INFO:teuthology.orchestra.run.smithi145.stdout:5219 punch from 0x2a14600 to 0x2a22c00, (0xe600 bytes) 2024-09-14T07:10:25.499 INFO:teuthology.orchestra.run.smithi145.stdout:5225 trunc from 0xc70f200 to 0x29e9a00 2024-09-14T07:10:25.526 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:25.523+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:25.544 INFO:teuthology.orchestra.run.smithi145.stdout:5226 read 0x1814600 thru 0x181bf61 (0x7962 bytes) 2024-09-14T07:10:25.578 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 128.275591921, 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:25.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:25.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:25.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 128.23060161500001, 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:25.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:25.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 123.92866915899999, 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:25.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:25.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 123.607341425, 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:25.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:25.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.170213025999999, 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.170202368000005, 2024-09-14T07:10:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.170187354000007, 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.170176439000002, 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.170164928999995, 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.170152385999998, 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163305789999995, 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163258009000003, 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163244542000001, 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163233570000003, 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163222808, 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163206754000001, 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163190182999998, 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163178407000004, 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.163167729999998, 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 23.001991305000001, 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:10:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:25.676 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:25.678 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5227 write 0x9775e00 thru 0x9783ab2 (0xdcb3 bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5228 punch from 0xe76600 to 0xe78a00, (0x2400 bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5229 read 0x50ea600 thru 0x50eb187 (0xb88 bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5234 read 0x3b94800 thru 0x3b9f000 (0xa801 bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5241 read 0x41e2400 thru 0x41eddeb (0xb9ec bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5244 read 0x46f4a00 thru 0x46fa81c (0x5e1d bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5245 read 0x4639c00 thru 0x4642548 (0x8949 bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5247 write 0x53ad400 thru 0x53b1352 (0x3f53 bytes) 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5248 trunc from 0x9783c00 to 0x8a2c800 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5249 trunc from 0x8a2c800 to 0x44b0600 2024-09-14T07:10:25.679 INFO:teuthology.orchestra.run.smithi145.stdout:5252 read 0x25bee00 thru 0x25ca1c9 (0xb3ca bytes) 2024-09-14T07:10:25.680 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:25.675+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:25.680 INFO:teuthology.orchestra.run.smithi145.stdout:5253 write 0x1b2a00 thru 0x1b9962 (0x6f63 bytes) 2024-09-14T07:10:25.685 INFO:teuthology.orchestra.run.smithi145.stdout:5254 trunc from 0x44b0600 to 0x6715400 2024-09-14T07:10:25.732 INFO:teuthology.orchestra.run.smithi145.stdout:5256 trunc from 0x6715400 to 0x6c7e00 2024-09-14T07:10:25.778 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:25.775+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:25.810 INFO:teuthology.orchestra.run.smithi145.stdout:5257 write 0xbdf4a00 thru 0xbdfe3a6 (0x99a7 bytes) 2024-09-14T07:10:25.817 INFO:teuthology.orchestra.run.smithi145.stdout:5258 trunc from 0xbdfe400 to 0x900a600 2024-09-14T07:10:25.832 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:25.832 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:25.832 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:25.832 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:25.849 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:25.853 INFO:teuthology.orchestra.run.smithi145.stdout:5260 read 0x6845600 thru 0x68545e8 (0xefe9 bytes) 2024-09-14T07:10:25.855 INFO:teuthology.orchestra.run.smithi145.stdout:5262 read 0x245bc00 thru 0x24624fb (0x68fc bytes) 2024-09-14T07:10:25.875 INFO:teuthology.orchestra.run.smithi145.stdout:5263 write 0xaa12e00 thru 0xaa174d1 (0x46d2 bytes) 2024-09-14T07:10:25.876 INFO:teuthology.orchestra.run.smithi145.stdout:5264 read 0x99f4200 thru 0x99f4d83 (0xb84 bytes) 2024-09-14T07:10:25.877 INFO:teuthology.orchestra.run.smithi145.stdout:5265 write 0x76f2600 thru 0x76f61a6 (0x3ba7 bytes) 2024-09-14T07:10:25.877 INFO:teuthology.orchestra.run.smithi145.stdout:5266 punch from 0x56a0600 to 0x56a3000, (0x2a00 bytes) 2024-09-14T07:10:25.877 INFO:teuthology.orchestra.run.smithi145.stdout:5267 read 0x7fdbc00 thru 0x7fe30de (0x74df bytes) 2024-09-14T07:10:25.877 INFO:teuthology.orchestra.run.smithi145.stdout:5268 read 0x2363c00 thru 0x2364c71 (0x1072 bytes) 2024-09-14T07:10:25.877 INFO:teuthology.orchestra.run.smithi145.stdout:5277 punch from 0x10ec400 to 0x10fa400, (0xe000 bytes) 2024-09-14T07:10:25.878 INFO:teuthology.orchestra.run.smithi145.stdout:5280 read 0xfc1400 thru 0xfc4398 (0x2f99 bytes) 2024-09-14T07:10:25.879 INFO:teuthology.orchestra.run.smithi145.stdout:5282 read 0x9c4f400 thru 0x9c5ab8b (0xb78c bytes) 2024-09-14T07:10:25.879 INFO:teuthology.orchestra.run.smithi145.stdout:5283 read 0x8572000 thru 0x8580756 (0xe757 bytes) 2024-09-14T07:10:25.879 INFO:teuthology.orchestra.run.smithi145.stdout:5284 read 0x5e24000 thru 0x5e3109c (0xd09d bytes) 2024-09-14T07:10:25.879 INFO:teuthology.orchestra.run.smithi145.stdout:5286 read 0x8530000 thru 0x8532fd9 (0x2fda bytes) 2024-09-14T07:10:25.879 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:25.879+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:25.886 INFO:teuthology.orchestra.run.smithi145.stdout:5289 trunc from 0xaa17600 to 0xded4600 2024-09-14T07:10:25.906 INFO:teuthology.orchestra.run.smithi145.stdout:5290 read 0x60a0a00 thru 0x60ad4ac (0xcaad bytes) 2024-09-14T07:10:25.907 INFO:teuthology.orchestra.run.smithi145.stdout:5291 write 0x75e4c00 thru 0x75f2cd9 (0xe0da bytes) 2024-09-14T07:10:25.908 INFO:teuthology.orchestra.run.smithi145.stdout:5292 read 0xa42e400 thru 0xa43554c (0x714d bytes) 2024-09-14T07:10:25.908 INFO:teuthology.orchestra.run.smithi145.stdout:5293 read 0xde23600 thru 0xde320dc (0xeadd bytes) 2024-09-14T07:10:25.908 INFO:teuthology.orchestra.run.smithi145.stdout:5295 read 0x8d45000 thru 0x8d50c0f (0xbc10 bytes) 2024-09-14T07:10:25.908 INFO:teuthology.orchestra.run.smithi145.stdout:5296 punch from 0xccc6800 to 0xccc8600, (0x1e00 bytes) 2024-09-14T07:10:25.909 INFO:teuthology.orchestra.run.smithi145.stdout:5297 write 0x6ec2200 thru 0x6ec79eb (0x57ec bytes) 2024-09-14T07:10:25.909 INFO:teuthology.orchestra.run.smithi145.stdout:5298 read 0x2220200 thru 0x222d603 (0xd404 bytes) 2024-09-14T07:10:25.910 INFO:teuthology.orchestra.run.smithi145.stdout:5300 write 0x1e2c200 thru 0x1e2f360 (0x3161 bytes) 2024-09-14T07:10:25.910 INFO:teuthology.orchestra.run.smithi145.stdout:5302 write 0x103fa00 thru 0x1040af4 (0x10f5 bytes) 2024-09-14T07:10:25.910 INFO:teuthology.orchestra.run.smithi145.stdout:5303 punch from 0x4eb4200 to 0x4ec2400, (0xe200 bytes) 2024-09-14T07:10:25.926 INFO:teuthology.orchestra.run.smithi145.stdout:5307 trunc from 0xded4600 to 0x64cce00 2024-09-14T07:10:25.950 INFO:teuthology.orchestra.run.smithi145.stdout:5308 read 0x392f800 thru 0x39384a6 (0x8ca7 bytes) 2024-09-14T07:10:25.951 INFO:teuthology.orchestra.run.smithi145.stdout:5309 read 0x2d82200 thru 0x2d8ffac (0xddad bytes) 2024-09-14T07:10:25.980 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:25.979+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:25.986 INFO:teuthology.orchestra.run.smithi145.stdout:5313 write 0xe169200 thru 0xe1702aa (0x70ab bytes) 2024-09-14T07:10:25.986 INFO:teuthology.orchestra.run.smithi145.stdout:5314 punch from 0x692400 to 0x69e200, (0xbe00 bytes) 2024-09-14T07:10:25.988 INFO:teuthology.orchestra.run.smithi145.stdout:5316 write 0x4ddc200 thru 0x4deb6ff (0xf500 bytes) 2024-09-14T07:10:25.988 INFO:teuthology.orchestra.run.smithi145.stdout:5317 write 0x675e00 thru 0x682219 (0xc41a bytes) 2024-09-14T07:10:26.001 INFO:teuthology.orchestra.run.smithi145.stdout:5318 trunc from 0xe170400 to 0xa25bc00 2024-09-14T07:10:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:10:26.021 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T07:10:26.042 INFO:teuthology.orchestra.run.smithi145.stdout:5319 read 0x9745c00 thru 0x974f5b2 (0x99b3 bytes) 2024-09-14T07:10:26.043 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:26.043 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:26.043 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:26.044 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:26.044 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:26.046 INFO:teuthology.orchestra.run.smithi145.stdout:5320 write 0x83bca00 thru 0x83cb6b8 (0xecb9 bytes) 2024-09-14T07:10:26.046 INFO:teuthology.orchestra.run.smithi145.stdout:5321 clone 38 order 25 su 16777216 sc 15 2024-09-14T07:10:26.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T07:10:26.061 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:26.095 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:26.095+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:26.181 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:26.179+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:26.183 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:26.183 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:26.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:26.183 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:26.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.183 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:26.183 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 173.87994254899999, 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:26.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:26.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.985949364999996, 2024-09-14T07:10:26.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:26.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:26.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:26.200 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.985829486, 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:26.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:26.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:26.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.968604365999994, 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:26.210 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:26.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.215 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 75.296009979000004, 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:26.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521968113, 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:10:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:10:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521935885000005, 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521913870000006, 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521892934999997, 2024-09-14T07:10:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521871924999999, 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521838224999996, 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521817581999997, 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521572364999997, 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521540646000005, 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521518886999999, 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.521486808000006, 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.520315601999997, 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.520292639000004, 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.520271348999998, 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.520251126000005, 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.298 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:26.300 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:26.300 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:26.299+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:26.400 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:26.399+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:26.477 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:26.477 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:26.477 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:26.477 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:26.492 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:26.502 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:26.499+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:26.603 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:26.599+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:26.615 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:26.615 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:26.616 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:26.629 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:26.704 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:26.703+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:26.758 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:26.758 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:26.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:26.758 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:26.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.758 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 129.45880828, 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:26.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.59933269299999, 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:26.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:26.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.599230408, 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:26.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:26.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 115.900178567, 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:26.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:26.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:26.779 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.415167006000004, 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:26.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:26.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.415132936999996, 2024-09-14T07:10:26.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:26.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:26.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.788 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.415089590999997, 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.415056067999998, 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.415008818000004, 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.414894570000001, 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.414784874999995, 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 80.414718731999997, 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.179642884, 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:10:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.892929448, 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:10:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.276914406, 2024-09-14T07:10:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:10:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:10:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:10:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:10:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.276850706999999, 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:10:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.276807373, 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.276750494, 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:10:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:10:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:10:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.276683998999999, 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.273799662, 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:10:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:10:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:10:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:10:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:10:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:26.858 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:26.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:26.860 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:26.861 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:26.859+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:26.960 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:26.959+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:27.014 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:27.014 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:27.017 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T07:10:27.040 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:27.040 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:27.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.040 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5950.0:2852 3.1d 3.a471f3d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-09-14T07:10:27.040 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:27.035674+0000", 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.001932752, 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019720340000000001, 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "op must wait for map", 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5950", 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2884569635", 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2852 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:27.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035674+0000", 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035674+0000", 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035676+0000", 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.773e-06 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035689+0000", 2024-09-14T07:10:27.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2203e-05 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035723+0000", 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4099e-05 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035740+0000", 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7557000000000001e-05 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035866+0000", 2024-09-14T07:10:27.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000125313 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:27.035876+0000", 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0543000000000001e-05 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:10:27.044 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:27.045 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:27.046 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T07:10:27.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:10:27.054 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:27.061 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:27.062 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:27.059+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:27.079 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:27.097 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:27.114 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:27.114 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:27.146 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone37 from 0xa25bc00 (overlap 0x6c7e00) to 0x15b200 2024-09-14T07:10:27.162 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:27.159+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:27.208 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:27.208 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:27.230 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:27.272 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:27.271+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:27.363 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:27.363+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:10:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:10:27.384 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-14T07:10:27.247959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1759155113,"num_objects":1389,"num_object_clones":238,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1389,"num_whiteouts":405,"num_read":301293,"num_read_kb":950301,"num_write":17879,"num_write_kb":304979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":172260973,"num_keys_recovered":413,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14636,"ondisk_log_size":14636,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":312596,"kb_used_data":49772,"kb_used_omap":7,"kb_used_meta":262712,"kb_avail":471546604,"statfs":{"total":483183820800,"available":482863722496,"internally_reserved":0,"allocated":50966528,"data_stored":378132917,"data_compressed":16625350,"data_compressed_allocated":34021376,"data_compressed_original":369401344,"omap_allocated":7973,"internal_metadata":269017307},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,57,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32111238,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":12744,"num_read_kb":41633,"num_write":1166,"num_write_kb":11683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000960"},"pg_stats":[{"pgid":"3.1f","version":"140'207","reported_seq":6168,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.153908+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:10:25.153908+0000","last_peered":"2024-09-14T07:10:25.153908+0000","last_clean":"2024-09-14T07:10:25.153908+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:10:25.153908+0000","last_undegraded":"2024-09-14T07:10:25.153908+0000","last_fullsized":"2024-09-14T07:10:25.153908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:24:49.687869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":74073616,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5455,"num_read_kb":46490,"num_write":307,"num_write_kb":11115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"137'517","reported_seq":10199,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289045+0000","last_change":"2024-09-14T07:10:12.821162+0000","last_active":"2024-09-14T07:10:25.289045+0000","last_peered":"2024-09-14T07:10:25.289045+0000","last_clean":"2024-09-14T07:10:25.289045+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:10:25.289045+0000","last_undegraded":"2024-09-14T07:10:25.289045+0000","last_fullsized":"2024-09-14T07:10:25.289045+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:36.122243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":70170502,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":18,"num_read":10990,"num_read_kb":46262,"num_write":800,"num_write_kb":14890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"140'1244","reported_seq":15168,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.278595+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:10:25.278595+0000","last_peered":"2024-09-14T07:10:25.278595+0000","last_clean":"2024-09-14T07:10:25.278595+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:10:25.278595+0000","last_undegraded":"2024-09-14T07:10:25.278595+0000","last_fullsized":"2024-09-14T07:10:25.278595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:01.517275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":14868778,"num_objects":48,"num_object_clones":4,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":18663,"num_read_kb":30296,"num_write":1235,"num_write_kb":4841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"140'290","reported_seq":18714,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289283+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:10:25.289283+0000","last_peered":"2024-09-14T07:10:25.289283+0000","last_clean":"2024-09-14T07:10:25.289283+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:10:25.289283+0000","last_undegraded":"2024-09-14T07:10:25.289283+0000","last_fullsized":"2024-09-14T07:10:25.289283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:16.820475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":43666484,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":9,"num_read":18254,"num_read_kb":40387,"num_write":305,"num_write_kb":7319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"140'454","reported_seq":8157,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.027034+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:10:25.027034+0000","last_peered":"2024-09-14T07:10:25.027034+0000","last_clean":"2024-09-14T07:10:25.027034+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:10:25.027034+0000","last_undegraded":"2024-09-14T07:10:25.027034+0000","last_fullsized":"2024-09-14T07:10:25.027034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:42:51.148497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":103910254,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":8427,"num_read_kb":21940,"num_write":587,"num_write_kb":10827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'312","reported_seq":28273,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.274069+0000","last_change":"2024-09-14T07:10:02.774511+0000","last_active":"2024-09-14T07:10:22.274069+0000","last_peered":"2024-09-14T07:10:22.274069+0000","last_clean":"2024-09-14T07:10:22.274069+0000","last_became_active":"2024-09-14T07:10:02.773571+0000","last_became_peered":"2024-09-14T07:10:02.773571+0000","last_unstale":"2024-09-14T07:10:22.274069+0000","last_undegraded":"2024-09-14T07:10:22.274069+0000","last_fullsized":"2024-09-14T07:10:22.274069+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:33:02.624669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":83192493,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":28552,"num_read_kb":48094,"num_write":366,"num_write_kb":11738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":27032787,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"137'1171","reported_seq":7380,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.251200+0000","last_change":"2024-09-14T07:10:02.775423+0000","last_active":"2024-09-14T07:10:22.251200+0000","last_peered":"2024-09-14T07:10:22.251200+0000","last_clean":"2024-09-14T07:10:22.251200+0000","last_became_active":"2024-09-14T07:10:02.773318+0000","last_became_peered":"2024-09-14T07:10:02.773318+0000","last_unstale":"2024-09-14T07:10:22.251200+0000","last_undegraded":"2024-09-14T07:10:22.251200+0000","last_fullsized":"2024-09-14T07:10:22.251200+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:12.238739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":35154196,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":15,"num_read":11160,"num_read_kb":25592,"num_write":2013,"num_write_kb":9095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":4196551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"137'329","reported_seq":4357,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.076824+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:10:25.076824+0000","last_peered":"2024-09-14T07:10:25.076824+0000","last_clean":"2024-09-14T07:10:25.076824+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:10:25.076824+0000","last_undegraded":"2024-09-14T07:10:25.076824+0000","last_fullsized":"2024-09-14T07:10:25.076824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:50:27.664499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":81316040,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":4085,"num_read_kb":11153,"num_write":411,"num_write_kb":14894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"137'204","reported_seq":3334,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040389+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:10:25.040389+0000","last_peered":"2024-09-14T07:10:25.040389+0000","last_clean":"2024-09-14T07:10:25.040389+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:10:25.040389+0000","last_undegraded":"2024-09-14T07:10:25.040389+0000","last_fullsized":"2024-09-14T07:10:25.040389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:50.471206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":41792550,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":6,"num_read":2748,"num_read_kb":19464,"num_write":218,"num_write_kb":4105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"137'523","reported_seq":8669,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.097609+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:10:25.097609+0000","last_peered":"2024-09-14T07:10:25.097609+0000","last_clean":"2024-09-14T07:10:25.097609+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:10:25.097609+0000","last_undegraded":"2024-09-14T07:10:25.097609+0000","last_fullsized":"2024-09-14T07:10:25.097609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:13:40.089670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":50965708,"num_objects":47,"num_object_clones":4,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":9792,"num_read_kb":37347,"num_write":629,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"137'355","reported_seq":4977,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289862+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:10:25.289862+0000","last_peered":"2024-09-14T07:10:25.289862+0000","last_clean":"2024-09-14T07:10:25.289862+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:10:25.289862+0000","last_undegraded":"2024-09-14T07:10:25.289862+0000","last_fullsized":"2024-09-14T07:10:25.289862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:22.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":70793265,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":5902,"num_read_kb":41713,"num_write":419,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'618","reported_seq":10031,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.222896+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:10:25.222896+0000","last_peered":"2024-09-14T07:10:25.222896+0000","last_clean":"2024-09-14T07:10:25.222896+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:10:25.222896+0000","last_undegraded":"2024-09-14T07:10:25.222896+0000","last_fullsized":"2024-09-14T07:10:25.222896+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:11:44.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":64520819,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":19,"num_read":11390,"num_read_kb":34963,"num_write":821,"num_write_kb":13696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.005066+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:10:25.005066+0000","last_peered":"2024-09-14T07:10:25.005066+0000","last_clean":"2024-09-14T07:10:25.005066+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:10:25.005066+0000","last_undegraded":"2024-09-14T07:10:25.005066+0000","last_fullsized":"2024-09-14T07:10:25.005066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'515","reported_seq":11141,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289296+0000","last_change":"2024-09-14T07:10:02.776247+0000","last_active":"2024-09-14T07:10:25.289296+0000","last_peered":"2024-09-14T07:10:25.289296+0000","last_clean":"2024-09-14T07:10:25.289296+0000","last_became_active":"2024-09-14T07:10:02.773321+0000","last_became_peered":"2024-09-14T07:10:02.773321+0000","last_unstale":"2024-09-14T07:10:25.289296+0000","last_undegraded":"2024-09-14T07:10:25.289296+0000","last_fullsized":"2024-09-14T07:10:25.289296+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:37:44.099807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":29248512,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":7,"num_read":13002,"num_read_kb":29607,"num_write":618,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":3295030,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004753+0000","last_change":"2024-09-14T07:10:00.564082+0000","last_active":"2024-09-14T07:10:25.004753+0000","last_peered":"2024-09-14T07:10:25.004753+0000","last_clean":"2024-09-14T07:10:25.004753+0000","last_became_active":"2024-09-14T07:10:00.563259+0000","last_became_peered":"2024-09-14T07:10:00.563259+0000","last_unstale":"2024-09-14T07:10:25.004753+0000","last_undegraded":"2024-09-14T07:10:25.004753+0000","last_fullsized":"2024-09-14T07:10:25.004753+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:00.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"137'432","reported_seq":5056,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.251483+0000","last_change":"2024-09-14T07:10:12.825814+0000","last_active":"2024-09-14T07:10:22.251483+0000","last_peered":"2024-09-14T07:10:22.251483+0000","last_clean":"2024-09-14T07:10:22.251483+0000","last_became_active":"2024-09-14T07:10:12.824726+0000","last_became_peered":"2024-09-14T07:10:12.824726+0000","last_unstale":"2024-09-14T07:10:22.251483+0000","last_undegraded":"2024-09-14T07:10:22.251483+0000","last_fullsized":"2024-09-14T07:10:22.251483+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:03.129989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":37214806,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5623,"num_read_kb":16651,"num_write":559,"num_write_kb":4011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":427094,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004679+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:10:25.004679+0000","last_peered":"2024-09-14T07:10:25.004679+0000","last_clean":"2024-09-14T07:10:25.004679+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:10:25.004679+0000","last_undegraded":"2024-09-14T07:10:25.004679+0000","last_fullsized":"2024-09-14T07:10:25.004679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"140'586","reported_seq":5753,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.156008+0000","last_change":"2024-09-14T07:10:12.821383+0000","last_active":"2024-09-14T07:10:25.156008+0000","last_peered":"2024-09-14T07:10:25.156008+0000","last_clean":"2024-09-14T07:10:25.156008+0000","last_became_active":"2024-09-14T07:10:12.820174+0000","last_became_peered":"2024-09-14T07:10:12.820174+0000","last_unstale":"2024-09-14T07:10:25.156008+0000","last_undegraded":"2024-09-14T07:10:25.156008+0000","last_fullsized":"2024-09-14T07:10:25.156008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:57.411785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":11951159,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":7111,"num_read_kb":13701,"num_write":785,"num_write_kb":3655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":492599,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:19.977403+0000","last_change":"2024-09-14T07:10:00.559864+0000","last_active":"2024-09-14T07:10:19.977403+0000","last_peered":"2024-09-14T07:10:19.977403+0000","last_clean":"2024-09-14T07:10:19.977403+0000","last_became_active":"2024-09-14T07:10:00.558149+0000","last_became_peered":"2024-09-14T07:10:00.558149+0000","last_unstale":"2024-09-14T07:10:19.977403+0000","last_undegraded":"2024-09-14T07:10:19.977403+0000","last_fullsized":"2024-09-14T07:10:19.977403+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:27.061139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"139'621","reported_seq":3151,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040506+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:10:25.040506+0000","last_peered":"2024-09-14T07:10:25.040506+0000","last_clean":"2024-09-14T07:10:25.040506+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:10:25.040506+0000","last_undegraded":"2024-09-14T07:10:25.040506+0000","last_fullsized":"2024-09-14T07:10:25.040506+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:43:42.932559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":16480557,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":4581,"num_read_kb":17849,"num_write":1006,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'234","reported_seq":8744,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.286227+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:10:22.286227+0000","last_peered":"2024-09-14T07:10:22.286227+0000","last_clean":"2024-09-14T07:10:22.286227+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:10:22.286227+0000","last_undegraded":"2024-09-14T07:10:22.286227+0000","last_fullsized":"2024-09-14T07:10:22.286227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:04:08.829536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":71780352,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":7885,"num_read_kb":44513,"num_write":307,"num_write_kb":16285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040134+0000","last_change":"2024-09-14T07:10:00.567828+0000","last_active":"2024-09-14T07:10:25.040134+0000","last_peered":"2024-09-14T07:10:25.040134+0000","last_clean":"2024-09-14T07:10:25.040134+0000","last_became_active":"2024-09-14T07:10:00.562043+0000","last_became_peered":"2024-09-14T07:10:00.562043+0000","last_unstale":"2024-09-14T07:10:25.040134+0000","last_undegraded":"2024-09-14T07:10:25.040134+0000","last_fullsized":"2024-09-14T07:10:25.040134+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:05:49.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"137'369","reported_seq":5038,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.138769+0000","last_change":"2024-09-14T07:10:12.826718+0000","last_active":"2024-09-14T07:10:22.138769+0000","last_peered":"2024-09-14T07:10:22.138769+0000","last_clean":"2024-09-14T07:10:22.138769+0000","last_became_active":"2024-09-14T07:10:12.824828+0000","last_became_peered":"2024-09-14T07:10:12.824828+0000","last_unstale":"2024-09-14T07:10:22.138769+0000","last_undegraded":"2024-09-14T07:10:22.138769+0000","last_fullsized":"2024-09-14T07:10:22.138769+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:13:03.893293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":33571207,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":12,"num_read":6650,"num_read_kb":20137,"num_write":297,"num_write_kb":3781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1876804,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.076842+0000","last_change":"2024-09-14T07:10:00.557817+0000","last_active":"2024-09-14T07:10:25.076842+0000","last_peered":"2024-09-14T07:10:25.076842+0000","last_clean":"2024-09-14T07:10:25.076842+0000","last_became_active":"2024-09-14T07:10:00.557388+0000","last_became_peered":"2024-09-14T07:10:00.557388+0000","last_unstale":"2024-09-14T07:10:25.076842+0000","last_undegraded":"2024-09-14T07:10:25.076842+0000","last_fullsized":"2024-09-14T07:10:25.076842+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:10.600536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"137'345","reported_seq":11658,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.308883+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:10:22.308883+0000","last_peered":"2024-09-14T07:10:22.308883+0000","last_clean":"2024-09-14T07:10:22.308883+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:10:22.308883+0000","last_undegraded":"2024-09-14T07:10:22.308883+0000","last_fullsized":"2024-09-14T07:10:22.308883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:57.893492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":22443561,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":11111,"num_read_kb":13714,"num_write":292,"num_write_kb":6584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003714+0000","last_change":"2024-09-14T07:10:00.561528+0000","last_active":"2024-09-14T07:10:25.003714+0000","last_peered":"2024-09-14T07:10:25.003714+0000","last_clean":"2024-09-14T07:10:25.003714+0000","last_became_active":"2024-09-14T07:10:00.560484+0000","last_became_peered":"2024-09-14T07:10:00.560484+0000","last_unstale":"2024-09-14T07:10:25.003714+0000","last_undegraded":"2024-09-14T07:10:25.003714+0000","last_fullsized":"2024-09-14T07:10:25.003714+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:16.642352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'277","reported_seq":5742,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.064297+0000","last_change":"2024-09-14T07:10:02.773083+0000","last_active":"2024-09-14T07:10:25.064297+0000","last_peered":"2024-09-14T07:10:25.064297+0000","last_clean":"2024-09-14T07:10:25.064297+0000","last_became_active":"2024-09-14T07:10:02.772053+0000","last_became_peered":"2024-09-14T07:10:02.772053+0000","last_unstale":"2024-09-14T07:10:25.064297+0000","last_undegraded":"2024-09-14T07:10:25.064297+0000","last_fullsized":"2024-09-14T07:10:25.064297+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:15.332776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":54619168,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5114,"num_read_kb":9947,"num_write":253,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":274464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":284,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040173+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:10:25.040173+0000","last_peered":"2024-09-14T07:10:25.040173+0000","last_clean":"2024-09-14T07:10:25.040173+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:10:25.040173+0000","last_undegraded":"2024-09-14T07:10:25.040173+0000","last_fullsized":"2024-09-14T07:10:25.040173+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":378,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003729+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:10:25.003729+0000","last_peered":"2024-09-14T07:10:25.003729+0000","last_clean":"2024-09-14T07:10:25.003729+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:10:25.003729+0000","last_undegraded":"2024-09-14T07:10:25.003729+0000","last_fullsized":"2024-09-14T07:10:25.003729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"140'360","reported_seq":9131,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289941+0000","last_change":"2024-09-14T07:10:12.824690+0000","last_active":"2024-09-14T07:10:25.289941+0000","last_peered":"2024-09-14T07:10:25.289941+0000","last_clean":"2024-09-14T07:10:25.289941+0000","last_became_active":"2024-09-14T07:10:12.823788+0000","last_became_peered":"2024-09-14T07:10:12.823788+0000","last_unstale":"2024-09-14T07:10:25.289941+0000","last_undegraded":"2024-09-14T07:10:25.289941+0000","last_fullsized":"2024-09-14T07:10:25.289941+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:55.176368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":150521428,"num_objects":56,"num_object_clones":9,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":11,"num_read":8632,"num_read_kb":63290,"num_write":498,"num_write_kb":27130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":14496852,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"137'545","reported_seq":6312,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.284469+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:10:22.284469+0000","last_peered":"2024-09-14T07:10:22.284469+0000","last_clean":"2024-09-14T07:10:22.284469+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:10:22.284469+0000","last_undegraded":"2024-09-14T07:10:22.284469+0000","last_fullsized":"2024-09-14T07:10:22.284469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:55.083685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":83919291,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":7417,"num_read_kb":43172,"num_write":624,"num_write_kb":16762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"138'324","reported_seq":6980,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.064338+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:10:25.064338+0000","last_peered":"2024-09-14T07:10:25.064338+0000","last_clean":"2024-09-14T07:10:25.064338+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:10:25.064338+0000","last_undegraded":"2024-09-14T07:10:25.064338+0000","last_fullsized":"2024-09-14T07:10:25.064338+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:06.304427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":36181008,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":6906,"num_read_kb":26946,"num_write":322,"num_write_kb":3473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'879","reported_seq":6947,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.297283+0000","last_change":"2024-09-14T07:10:02.776185+0000","last_active":"2024-09-14T07:10:22.297283+0000","last_peered":"2024-09-14T07:10:22.297283+0000","last_clean":"2024-09-14T07:10:22.297283+0000","last_became_active":"2024-09-14T07:10:02.774294+0000","last_became_peered":"2024-09-14T07:10:02.774294+0000","last_unstale":"2024-09-14T07:10:22.297283+0000","last_undegraded":"2024-09-14T07:10:22.297283+0000","last_fullsized":"2024-09-14T07:10:22.297283+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:16.895719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":66279755,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":12,"num_read":9379,"num_read_kb":36398,"num_write":1262,"num_write_kb":13591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":25856356,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"140'406","reported_seq":14552,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040755+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:10:25.040755+0000","last_peered":"2024-09-14T07:10:25.040755+0000","last_clean":"2024-09-14T07:10:25.040755+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:10:25.040755+0000","last_undegraded":"2024-09-14T07:10:25.040755+0000","last_fullsized":"2024-09-14T07:10:25.040755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:57.711682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":55193600,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":15750,"num_read_kb":34800,"num_write":319,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"140'284","reported_seq":3543,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.098660+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:10:25.098660+0000","last_peered":"2024-09-14T07:10:25.098660+0000","last_clean":"2024-09-14T07:10:25.098660+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:10:25.098660+0000","last_undegraded":"2024-09-14T07:10:25.098660+0000","last_fullsized":"2024-09-14T07:10:25.098660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:43.076004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":32752340,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":3164,"num_read_kb":17058,"num_write":285,"num_write_kb":4854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'376","reported_seq":16405,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289760+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:10:25.289760+0000","last_peered":"2024-09-14T07:10:25.289760+0000","last_clean":"2024-09-14T07:10:25.289760+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:10:25.289760+0000","last_undegraded":"2024-09-14T07:10:25.289760+0000","last_fullsized":"2024-09-14T07:10:25.289760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:32:53.814868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":44772972,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":17138,"num_read_kb":36275,"num_write":379,"num_write_kb":5774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"140'367","reported_seq":7654,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.207901+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:10:25.207901+0000","last_peered":"2024-09-14T07:10:25.207901+0000","last_clean":"2024-09-14T07:10:25.207901+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:10:25.207901+0000","last_undegraded":"2024-09-14T07:10:25.207901+0000","last_fullsized":"2024-09-14T07:10:25.207901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:48:10.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":32828057,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":7877,"num_read_kb":13306,"num_write":523,"num_write_kb":16586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"137'259","reported_seq":6884,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.222143+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:10:25.222143+0000","last_peered":"2024-09-14T07:10:25.222143+0000","last_clean":"2024-09-14T07:10:25.222143+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:10:25.222143+0000","last_undegraded":"2024-09-14T07:10:25.222143+0000","last_fullsized":"2024-09-14T07:10:25.222143+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:58:00.176170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":56167714,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":6061,"num_read_kb":22656,"num_write":360,"num_write_kb":16646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"140'448","reported_seq":8573,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289218+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:10:25.289218+0000","last_peered":"2024-09-14T07:10:25.289218+0000","last_clean":"2024-09-14T07:10:25.289218+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:10:25.289218+0000","last_undegraded":"2024-09-14T07:10:25.289218+0000","last_fullsized":"2024-09-14T07:10:25.289218+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:04:49.483505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":60875407,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":9856,"num_read_kb":20615,"num_write":350,"num_write_kb":8566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"140'426","reported_seq":6321,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004276+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:10:25.004276+0000","last_peered":"2024-09-14T07:10:25.004276+0000","last_clean":"2024-09-14T07:10:25.004276+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:10:25.004276+0000","last_undegraded":"2024-09-14T07:10:25.004276+0000","last_fullsized":"2024-09-14T07:10:25.004276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:13.059416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":59173443,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":6978,"num_read_kb":21526,"num_write":340,"num_write_kb":7206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"137'284","reported_seq":4333,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003969+0000","last_change":"2024-09-14T07:10:02.772027+0000","last_active":"2024-09-14T07:10:25.003969+0000","last_peered":"2024-09-14T07:10:25.003969+0000","last_clean":"2024-09-14T07:10:25.003969+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:10:25.003969+0000","last_undegraded":"2024-09-14T07:10:25.003969+0000","last_fullsized":"2024-09-14T07:10:25.003969+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:48:10.697379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":68165684,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":5574,"num_read_kb":44384,"num_write":286,"num_write_kb":5176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1758564726,"num_objects":1385,"num_object_clones":238,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":405,"num_read":301227,"num_read_kb":950246,"num_write":17776,"num_write_kb":303349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1555,"num_bytes_recovered":171080218,"num_keys_recovered":413,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48246784,"data_stored":433982763,"data_compressed":18579109,"data_compressed_allocated":38580224,"data_compressed_original":428144128,"omap_allocated":0,"internal_metadata":0},"log_size":14561,"ondisk_log_size":14561,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79424,"kb_used_data":12144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94292416,"statfs":{"total":96636764160,"available":96555433984,"internally_reserved":0,"allocated":12435456,"data_stored":91100714,"data_compressed":4338208,"data_compressed_allocated":8634368,"data_compressed_original":89143296,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,57,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607591,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61724,"kb_used_data":8324,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310116,"statfs":{"total":96636764160,"available":96573558784,"internally_reserved":0,"allocated":8523776,"data_stored":52390853,"data_compressed":2987999,"data_compressed_allocated":5382144,"data_compressed_original":50787840,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67508,"kb_used_data":9948,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304332,"statfs":{"total":96636764160,"available":96567635968,"internally_reserved":0,"allocated":10186752,"data_stored":77394047,"data_compressed":3289172,"data_compressed_allocated":6742016,"data_compressed_original":75619840,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607592,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62588,"kb_used_data":9192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309252,"statfs":{"total":96636764160,"available":96572674048,"internally_reserved":0,"allocated":9412608,"data_stored":74217174,"data_compressed":2916331,"data_compressed_allocated":6201344,"data_compressed_original":72620544,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607592,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607592,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41352,"kb_used_data":10164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":10407936,"data_stored":83030129,"data_compressed":3093640,"data_compressed_allocated":7061504,"data_compressed_original":81229824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8728576,"data_stored":81789318,"data_compressed":3102932,"data_compressed_allocated":7041024,"data_compressed_original":80664576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5402624,"data_stored":60083276,"data_compressed":1820382,"data_compressed_allocated":4485120,"data_compressed_original":59631616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7819264,"data_stored":73655659,"data_compressed":2946347,"data_compressed_allocated":6242304,"data_compressed_original":72722944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8560640,"data_stored":76230388,"data_compressed":3329296,"data_compressed_allocated":6758400,"data_compressed_original":75136512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":90379931,"data_compressed":4337914,"data_compressed_allocated":8605696,"data_compressed_original":89085952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6959104,"data_stored":51844191,"data_compressed":3042238,"data_compressed_allocated":5447680,"data_compressed_original":50902528,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:10:27.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:10:27.396 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:27.396 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:27.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:27.471 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:27.471+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:27.550 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:27.550 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:27.565 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:27.563+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:27.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:27.666 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:27.663+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:27.726 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:27.726 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:27.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:27.745 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:10:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:10:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-14T07:10:27.247959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1759155113,"num_objects":1389,"num_object_clones":238,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1389,"num_whiteouts":405,"num_read":301293,"num_read_kb":950301,"num_write":17879,"num_write_kb":304979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":172260973,"num_keys_recovered":413,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14636,"ondisk_log_size":14636,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":312596,"kb_used_data":49772,"kb_used_omap":7,"kb_used_meta":262712,"kb_avail":471546604,"statfs":{"total":483183820800,"available":482863722496,"internally_reserved":0,"allocated":50966528,"data_stored":378132917,"data_compressed":16625350,"data_compressed_allocated":34021376,"data_compressed_original":369401344,"omap_allocated":7973,"internal_metadata":269017307},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,57,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32111238,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":12744,"num_read_kb":41633,"num_write":1166,"num_write_kb":11683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000960"},"pg_stats":[{"pgid":"3.1f","version":"140'207","reported_seq":6168,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.153908+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:10:25.153908+0000","last_peered":"2024-09-14T07:10:25.153908+0000","last_clean":"2024-09-14T07:10:25.153908+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:10:25.153908+0000","last_undegraded":"2024-09-14T07:10:25.153908+0000","last_fullsized":"2024-09-14T07:10:25.153908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:24:49.687869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":74073616,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5455,"num_read_kb":46490,"num_write":307,"num_write_kb":11115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"137'517","reported_seq":10199,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289045+0000","last_change":"2024-09-14T07:10:12.821162+0000","last_active":"2024-09-14T07:10:25.289045+0000","last_peered":"2024-09-14T07:10:25.289045+0000","last_clean":"2024-09-14T07:10:25.289045+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:10:25.289045+0000","last_undegraded":"2024-09-14T07:10:25.289045+0000","last_fullsized":"2024-09-14T07:10:25.289045+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:36.122243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":70170502,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":18,"num_read":10990,"num_read_kb":46262,"num_write":800,"num_write_kb":14890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"140'1244","reported_seq":15168,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.278595+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:10:25.278595+0000","last_peered":"2024-09-14T07:10:25.278595+0000","last_clean":"2024-09-14T07:10:25.278595+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:10:25.278595+0000","last_undegraded":"2024-09-14T07:10:25.278595+0000","last_fullsized":"2024-09-14T07:10:25.278595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:01.517275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":14868778,"num_objects":48,"num_object_clones":4,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":18663,"num_read_kb":30296,"num_write":1235,"num_write_kb":4841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"140'290","reported_seq":18714,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289283+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:10:25.289283+0000","last_peered":"2024-09-14T07:10:25.289283+0000","last_clean":"2024-09-14T07:10:25.289283+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:10:25.289283+0000","last_undegraded":"2024-09-14T07:10:25.289283+0000","last_fullsized":"2024-09-14T07:10:25.289283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:16.820475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":43666484,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":9,"num_read":18254,"num_read_kb":40387,"num_write":305,"num_write_kb":7319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"140'454","reported_seq":8157,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.027034+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:10:25.027034+0000","last_peered":"2024-09-14T07:10:25.027034+0000","last_clean":"2024-09-14T07:10:25.027034+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:10:25.027034+0000","last_undegraded":"2024-09-14T07:10:25.027034+0000","last_fullsized":"2024-09-14T07:10:25.027034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:42:51.148497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":103910254,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":8427,"num_read_kb":21940,"num_write":587,"num_write_kb":10827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'312","reported_seq":28273,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.274069+0000","last_change":"2024-09-14T07:10:02.774511+0000","last_active":"2024-09-14T07:10:22.274069+0000","last_peered":"2024-09-14T07:10:22.274069+0000","last_clean":"2024-09-14T07:10:22.274069+0000","last_became_active":"2024-09-14T07:10:02.773571+0000","last_became_peered":"2024-09-14T07:10:02.773571+0000","last_unstale":"2024-09-14T07:10:22.274069+0000","last_undegraded":"2024-09-14T07:10:22.274069+0000","last_fullsized":"2024-09-14T07:10:22.274069+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:33:02.624669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":83192493,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":28552,"num_read_kb":48094,"num_write":366,"num_write_kb":11738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":27032787,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"137'1171","reported_seq":7380,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.251200+0000","last_change":"2024-09-14T07:10:02.775423+0000","last_active":"2024-09-14T07:10:22.251200+0000","last_peered":"2024-09-14T07:10:22.251200+0000","last_clean":"2024-09-14T07:10:22.251200+0000","last_became_active":"2024-09-14T07:10:02.773318+0000","last_became_peered":"2024-09-14T07:10:02.773318+0000","last_unstale":"2024-09-14T07:10:22.251200+0000","last_undegraded":"2024-09-14T07:10:22.251200+0000","last_fullsized":"2024-09-14T07:10:22.251200+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:12.238739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":35154196,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":15,"num_read":11160,"num_read_kb":25592,"num_write":2013,"num_write_kb":9095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":4196551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"137'329","reported_seq":4357,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.076824+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:10:25.076824+0000","last_peered":"2024-09-14T07:10:25.076824+0000","last_clean":"2024-09-14T07:10:25.076824+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:10:25.076824+0000","last_undegraded":"2024-09-14T07:10:25.076824+0000","last_fullsized":"2024-09-14T07:10:25.076824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:50:27.664499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":81316040,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":4085,"num_read_kb":11153,"num_write":411,"num_write_kb":14894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"137'204","reported_seq":3334,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040389+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:10:25.040389+0000","last_peered":"2024-09-14T07:10:25.040389+0000","last_clean":"2024-09-14T07:10:25.040389+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:10:25.040389+0000","last_undegraded":"2024-09-14T07:10:25.040389+0000","last_fullsized":"2024-09-14T07:10:25.040389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:50.471206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":41792550,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":6,"num_read":2748,"num_read_kb":19464,"num_write":218,"num_write_kb":4105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"137'523","reported_seq":8669,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.097609+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:10:25.097609+0000","last_peered":"2024-09-14T07:10:25.097609+0000","last_clean":"2024-09-14T07:10:25.097609+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:10:25.097609+0000","last_undegraded":"2024-09-14T07:10:25.097609+0000","last_fullsized":"2024-09-14T07:10:25.097609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:13:40.089670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":50965708,"num_objects":47,"num_object_clones":4,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":9792,"num_read_kb":37347,"num_write":629,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"137'355","reported_seq":4977,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289862+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:10:25.289862+0000","last_peered":"2024-09-14T07:10:25.289862+0000","last_clean":"2024-09-14T07:10:25.289862+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:10:25.289862+0000","last_undegraded":"2024-09-14T07:10:25.289862+0000","last_fullsized":"2024-09-14T07:10:25.289862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:22.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":70793265,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":5902,"num_read_kb":41713,"num_write":419,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'618","reported_seq":10031,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.222896+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:10:25.222896+0000","last_peered":"2024-09-14T07:10:25.222896+0000","last_clean":"2024-09-14T07:10:25.222896+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:10:25.222896+0000","last_undegraded":"2024-09-14T07:10:25.222896+0000","last_fullsized":"2024-09-14T07:10:25.222896+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:11:44.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":64520819,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":19,"num_read":11390,"num_read_kb":34963,"num_write":821,"num_write_kb":13696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.005066+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:10:25.005066+0000","last_peered":"2024-09-14T07:10:25.005066+0000","last_clean":"2024-09-14T07:10:25.005066+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:10:25.005066+0000","last_undegraded":"2024-09-14T07:10:25.005066+0000","last_fullsized":"2024-09-14T07:10:25.005066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'515","reported_seq":11141,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289296+0000","last_change":"2024-09-14T07:10:02.776247+0000","last_active":"2024-09-14T07:10:25.289296+0000","last_peered":"2024-09-14T07:10:25.289296+0000","last_clean":"2024-09-14T07:10:25.289296+0000","last_became_active":"2024-09-14T07:10:02.773321+0000","last_became_peered":"2024-09-14T07:10:02.773321+0000","last_unstale":"2024-09-14T07:10:25.289296+0000","last_undegraded":"2024-09-14T07:10:25.289296+0000","last_fullsized":"2024-09-14T07:10:25.289296+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:37:44.099807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":29248512,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":7,"num_read":13002,"num_read_kb":29607,"num_write":618,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":3295030,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004753+0000","last_change":"2024-09-14T07:10:00.564082+0000","last_active":"2024-09-14T07:10:25.004753+0000","last_peered":"2024-09-14T07:10:25.004753+0000","last_clean":"2024-09-14T07:10:25.004753+0000","last_became_active":"2024-09-14T07:10:00.563259+0000","last_became_peered":"2024-09-14T07:10:00.563259+0000","last_unstale":"2024-09-14T07:10:25.004753+0000","last_undegraded":"2024-09-14T07:10:25.004753+0000","last_fullsized":"2024-09-14T07:10:25.004753+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:00.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"137'432","reported_seq":5056,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.251483+0000","last_change":"2024-09-14T07:10:12.825814+0000","last_active":"2024-09-14T07:10:22.251483+0000","last_peered":"2024-09-14T07:10:22.251483+0000","last_clean":"2024-09-14T07:10:22.251483+0000","last_became_active":"2024-09-14T07:10:12.824726+0000","last_became_peered":"2024-09-14T07:10:12.824726+0000","last_unstale":"2024-09-14T07:10:22.251483+0000","last_undegraded":"2024-09-14T07:10:22.251483+0000","last_fullsized":"2024-09-14T07:10:22.251483+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:03.129989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":37214806,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5623,"num_read_kb":16651,"num_write":559,"num_write_kb":4011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":427094,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004679+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:10:25.004679+0000","last_peered":"2024-09-14T07:10:25.004679+0000","last_clean":"2024-09-14T07:10:25.004679+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:10:25.004679+0000","last_undegraded":"2024-09-14T07:10:25.004679+0000","last_fullsized":"2024-09-14T07:10:25.004679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"140'586","reported_seq":5753,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.156008+0000","last_change":"2024-09-14T07:10:12.821383+0000","last_active":"2024-09-14T07:10:25.156008+0000","last_peered":"2024-09-14T07:10:25.156008+0000","last_clean":"2024-09-14T07:10:25.156008+0000","last_became_active":"2024-09-14T07:10:12.820174+0000","last_became_peered":"2024-09-14T07:10:12.820174+0000","last_unstale":"2024-09-14T07:10:25.156008+0000","last_undegraded":"2024-09-14T07:10:25.156008+0000","last_fullsized":"2024-09-14T07:10:25.156008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:57.411785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":11951159,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":7111,"num_read_kb":13701,"num_write":785,"num_write_kb":3655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":492599,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:19.977403+0000","last_change":"2024-09-14T07:10:00.559864+0000","last_active":"2024-09-14T07:10:19.977403+0000","last_peered":"2024-09-14T07:10:19.977403+0000","last_clean":"2024-09-14T07:10:19.977403+0000","last_became_active":"2024-09-14T07:10:00.558149+0000","last_became_peered":"2024-09-14T07:10:00.558149+0000","last_unstale":"2024-09-14T07:10:19.977403+0000","last_undegraded":"2024-09-14T07:10:19.977403+0000","last_fullsized":"2024-09-14T07:10:19.977403+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:27.061139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"139'621","reported_seq":3151,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040506+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:10:25.040506+0000","last_peered":"2024-09-14T07:10:25.040506+0000","last_clean":"2024-09-14T07:10:25.040506+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:10:25.040506+0000","last_undegraded":"2024-09-14T07:10:25.040506+0000","last_fullsized":"2024-09-14T07:10:25.040506+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:43:42.932559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":16480557,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":4581,"num_read_kb":17849,"num_write":1006,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'234","reported_seq":8744,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.286227+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:10:22.286227+0000","last_peered":"2024-09-14T07:10:22.286227+0000","last_clean":"2024-09-14T07:10:22.286227+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:10:22.286227+0000","last_undegraded":"2024-09-14T07:10:22.286227+0000","last_fullsized":"2024-09-14T07:10:22.286227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:04:08.829536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":71780352,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":7885,"num_read_kb":44513,"num_write":307,"num_write_kb":16285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040134+0000","last_change":"2024-09-14T07:10:00.567828+0000","last_active":"2024-09-14T07:10:25.040134+0000","last_peered":"2024-09-14T07:10:25.040134+0000","last_clean":"2024-09-14T07:10:25.040134+0000","last_became_active":"2024-09-14T07:10:00.562043+0000","last_became_peered":"2024-09-14T07:10:00.562043+0000","last_unstale":"2024-09-14T07:10:25.040134+0000","last_undegraded":"2024-09-14T07:10:25.040134+0000","last_fullsized":"2024-09-14T07:10:25.040134+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:05:49.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"137'369","reported_seq":5038,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.138769+0000","last_change":"2024-09-14T07:10:12.826718+0000","last_active":"2024-09-14T07:10:22.138769+0000","last_peered":"2024-09-14T07:10:22.138769+0000","last_clean":"2024-09-14T07:10:22.138769+0000","last_became_active":"2024-09-14T07:10:12.824828+0000","last_became_peered":"2024-09-14T07:10:12.824828+0000","last_unstale":"2024-09-14T07:10:22.138769+0000","last_undegraded":"2024-09-14T07:10:22.138769+0000","last_fullsized":"2024-09-14T07:10:22.138769+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:13:03.893293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":33571207,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":12,"num_read":6650,"num_read_kb":20137,"num_write":297,"num_write_kb":3781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1876804,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.076842+0000","last_change":"2024-09-14T07:10:00.557817+0000","last_active":"2024-09-14T07:10:25.076842+0000","last_peered":"2024-09-14T07:10:25.076842+0000","last_clean":"2024-09-14T07:10:25.076842+0000","last_became_active":"2024-09-14T07:10:00.557388+0000","last_became_peered":"2024-09-14T07:10:00.557388+0000","last_unstale":"2024-09-14T07:10:25.076842+0000","last_undegraded":"2024-09-14T07:10:25.076842+0000","last_fullsized":"2024-09-14T07:10:25.076842+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:10.600536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"137'345","reported_seq":11658,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.308883+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:10:22.308883+0000","last_peered":"2024-09-14T07:10:22.308883+0000","last_clean":"2024-09-14T07:10:22.308883+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:10:22.308883+0000","last_undegraded":"2024-09-14T07:10:22.308883+0000","last_fullsized":"2024-09-14T07:10:22.308883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:57.893492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":22443561,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":11111,"num_read_kb":13714,"num_write":292,"num_write_kb":6584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003714+0000","last_change":"2024-09-14T07:10:00.561528+0000","last_active":"2024-09-14T07:10:25.003714+0000","last_peered":"2024-09-14T07:10:25.003714+0000","last_clean":"2024-09-14T07:10:25.003714+0000","last_became_active":"2024-09-14T07:10:00.560484+0000","last_became_peered":"2024-09-14T07:10:00.560484+0000","last_unstale":"2024-09-14T07:10:25.003714+0000","last_undegraded":"2024-09-14T07:10:25.003714+0000","last_fullsized":"2024-09-14T07:10:25.003714+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:16.642352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'277","reported_seq":5742,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.064297+0000","last_change":"2024-09-14T07:10:02.773083+0000","last_active":"2024-09-14T07:10:25.064297+0000","last_peered":"2024-09-14T07:10:25.064297+0000","last_clean":"2024-09-14T07:10:25.064297+0000","last_became_active":"2024-09-14T07:10:02.772053+0000","last_became_peered":"2024-09-14T07:10:02.772053+0000","last_unstale":"2024-09-14T07:10:25.064297+0000","last_undegraded":"2024-09-14T07:10:25.064297+0000","last_fullsized":"2024-09-14T07:10:25.064297+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:15.332776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":54619168,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5114,"num_read_kb":9947,"num_write":253,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":274464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":284,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040173+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:10:25.040173+0000","last_peered":"2024-09-14T07:10:25.040173+0000","last_clean":"2024-09-14T07:10:25.040173+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:10:25.040173+0000","last_undegraded":"2024-09-14T07:10:25.040173+0000","last_fullsized":"2024-09-14T07:10:25.040173+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":378,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003729+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:10:25.003729+0000","last_peered":"2024-09-14T07:10:25.003729+0000","last_clean":"2024-09-14T07:10:25.003729+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:10:25.003729+0000","last_undegraded":"2024-09-14T07:10:25.003729+0000","last_fullsized":"2024-09-14T07:10:25.003729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"140'360","reported_seq":9131,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289941+0000","last_change":"2024-09-14T07:10:12.824690+0000","last_active":"2024-09-14T07:10:25.289941+0000","last_peered":"2024-09-14T07:10:25.289941+0000","last_clean":"2024-09-14T07:10:25.289941+0000","last_became_active":"2024-09-14T07:10:12.823788+0000","last_became_peered":"2024-09-14T07:10:12.823788+0000","last_unstale":"2024-09-14T07:10:25.289941+0000","last_undegraded":"2024-09-14T07:10:25.289941+0000","last_fullsized":"2024-09-14T07:10:25.289941+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:55.176368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":150521428,"num_objects":56,"num_object_clones":9,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":11,"num_read":8632,"num_read_kb":63290,"num_write":498,"num_write_kb":27130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":14496852,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"137'545","reported_seq":6312,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.284469+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:10:22.284469+0000","last_peered":"2024-09-14T07:10:22.284469+0000","last_clean":"2024-09-14T07:10:22.284469+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:10:22.284469+0000","last_undegraded":"2024-09-14T07:10:22.284469+0000","last_fullsized":"2024-09-14T07:10:22.284469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:55.083685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":83919291,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":7417,"num_read_kb":43172,"num_write":624,"num_write_kb":16762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"138'324","reported_seq":6980,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.064338+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:10:25.064338+0000","last_peered":"2024-09-14T07:10:25.064338+0000","last_clean":"2024-09-14T07:10:25.064338+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:10:25.064338+0000","last_undegraded":"2024-09-14T07:10:25.064338+0000","last_fullsized":"2024-09-14T07:10:25.064338+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:06.304427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":36181008,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":6906,"num_read_kb":26946,"num_write":322,"num_write_kb":3473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'879","reported_seq":6947,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.297283+0000","last_change":"2024-09-14T07:10:02.776185+0000","last_active":"2024-09-14T07:10:22.297283+0000","last_peered":"2024-09-14T07:10:22.297283+0000","last_clean":"2024-09-14T07:10:22.297283+0000","last_became_active":"2024-09-14T07:10:02.774294+0000","last_became_peered":"2024-09-14T07:10:02.774294+0000","last_unstale":"2024-09-14T07:10:22.297283+0000","last_undegraded":"2024-09-14T07:10:22.297283+0000","last_fullsized":"2024-09-14T07:10:22.297283+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:16.895719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":66279755,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":12,"num_read":9379,"num_read_kb":36398,"num_write":1262,"num_write_kb":13591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":25856356,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"140'406","reported_seq":14552,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040755+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:10:25.040755+0000","last_peered":"2024-09-14T07:10:25.040755+0000","last_clean":"2024-09-14T07:10:25.040755+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:10:25.040755+0000","last_undegraded":"2024-09-14T07:10:25.040755+0000","last_fullsized":"2024-09-14T07:10:25.040755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:57.711682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":55193600,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":15750,"num_read_kb":34800,"num_write":319,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"140'284","reported_seq":3543,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.098660+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:10:25.098660+0000","last_peered":"2024-09-14T07:10:25.098660+0000","last_clean":"2024-09-14T07:10:25.098660+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:10:25.098660+0000","last_undegraded":"2024-09-14T07:10:25.098660+0000","last_fullsized":"2024-09-14T07:10:25.098660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:43.076004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":32752340,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":3164,"num_read_kb":17058,"num_write":285,"num_write_kb":4854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'376","reported_seq":16405,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289760+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:10:25.289760+0000","last_peered":"2024-09-14T07:10:25.289760+0000","last_clean":"2024-09-14T07:10:25.289760+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:10:25.289760+0000","last_undegraded":"2024-09-14T07:10:25.289760+0000","last_fullsized":"2024-09-14T07:10:25.289760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:32:53.814868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":44772972,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":17138,"num_read_kb":36275,"num_write":379,"num_write_kb":5774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"140'367","reported_seq":7654,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.207901+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:10:25.207901+0000","last_peered":"2024-09-14T07:10:25.207901+0000","last_clean":"2024-09-14T07:10:25.207901+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:10:25.207901+0000","last_undegraded":"2024-09-14T07:10:25.207901+0000","last_fullsized":"2024-09-14T07:10:25.207901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:48:10.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":32828057,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":7877,"num_read_kb":13306,"num_write":523,"num_write_kb":16586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"137'259","reported_seq":6884,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.222143+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:10:25.222143+0000","last_peered":"2024-09-14T07:10:25.222143+0000","last_clean":"2024-09-14T07:10:25.222143+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:10:25.222143+0000","last_undegraded":"2024-09-14T07:10:25.222143+0000","last_fullsized":"2024-09-14T07:10:25.222143+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:58:00.176170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":56167714,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":6061,"num_read_kb":22656,"num_write":360,"num_write_kb":16646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"140'448","reported_seq":8573,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289218+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:10:25.289218+0000","last_peered":"2024-09-14T07:10:25.289218+0000","last_clean":"2024-09-14T07:10:25.289218+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:10:25.289218+0000","last_undegraded":"2024-09-14T07:10:25.289218+0000","last_fullsized":"2024-09-14T07:10:25.289218+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:04:49.483505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":60875407,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":9856,"num_read_kb":20615,"num_write":350,"num_write_kb":8566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"140'426","reported_seq":6321,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004276+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:10:25.004276+0000","last_peered":"2024-09-14T07:10:25.004276+0000","last_clean":"2024-09-14T07:10:25.004276+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:10:25.004276+0000","last_undegraded":"2024-09-14T07:10:25.004276+0000","last_fullsized":"2024-09-14T07:10:25.004276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:13.059416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":59173443,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":6978,"num_read_kb":21526,"num_write":340,"num_write_kb":7206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"137'284","reported_seq":4333,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003969+0000","last_change":"2024-09-14T07:10:02.772027+0000","last_active":"2024-09-14T07:10:25.003969+0000","last_peered":"2024-09-14T07:10:25.003969+0000","last_clean":"2024-09-14T07:10:25.003969+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:10:25.003969+0000","last_undegraded":"2024-09-14T07:10:25.003969+0000","last_fullsized":"2024-09-14T07:10:25.003969+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:48:10.697379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":68165684,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":5574,"num_read_kb":44384,"num_write":286,"num_write_kb":5176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1758564726,"num_objects":1385,"num_object_clones":238,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":405,"num_read":301227,"num_read_kb":950246,"num_write":17776,"num_write_kb":303349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1555,"num_bytes_recovered":171080218,"num_keys_recovered":413,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48246784,"data_stored":433982763,"data_compressed":18579109,"data_compressed_allocated":38580224,"data_compressed_original":428144128,"omap_allocated":0,"internal_metadata":0},"log_size":14561,"ondisk_log_size":14561,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79424,"kb_used_data":12144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94292416,"statfs":{"total":96636764160,"available":96555433984,"internally_reserved":0,"allocated":12435456,"data_stored":91100714,"data_compressed":4338208,"data_compressed_allocated":8634368,"data_compressed_original":89143296,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,57,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607591,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61724,"kb_used_data":8324,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310116,"statfs":{"total":96636764160,"available":96573558784,"internally_reserved":0,"allocated":8523776,"data_stored":52390853,"data_compressed":2987999,"data_compressed_allocated":5382144,"data_compressed_original":50787840,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67508,"kb_used_data":9948,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304332,"statfs":{"total":96636764160,"available":96567635968,"internally_reserved":0,"allocated":10186752,"data_stored":77394047,"data_compressed":3289172,"data_compressed_allocated":6742016,"data_compressed_original":75619840,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607592,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62588,"kb_used_data":9192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309252,"statfs":{"total":96636764160,"available":96572674048,"internally_reserved":0,"allocated":9412608,"data_stored":74217174,"data_compressed":2916331,"data_compressed_allocated":6201344,"data_compressed_original":72620544,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607592,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607592,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41352,"kb_used_data":10164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":10407936,"data_stored":83030129,"data_compressed":3093640,"data_compressed_allocated":7061504,"data_compressed_original":81229824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8728576,"data_stored":81789318,"data_compressed":3102932,"data_compressed_allocated":7041024,"data_compressed_original":80664576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5402624,"data_stored":60083276,"data_compressed":1820382,"data_compressed_allocated":4485120,"data_compressed_original":59631616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7819264,"data_stored":73655659,"data_compressed":2946347,"data_compressed_allocated":6242304,"data_compressed_original":72722944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8560640,"data_stored":76230388,"data_compressed":3329296,"data_compressed_allocated":6758400,"data_compressed_original":75136512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":90379931,"data_compressed":4337914,"data_compressed_allocated":8605696,"data_compressed_original":89085952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6959104,"data_stored":51844191,"data_compressed":3042238,"data_compressed_allocated":5447680,"data_compressed_original":50902528,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:10:27.767 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T07:10:27.767 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:27.768 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T07:10:27.768 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:10:27.769 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:27.767+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:27.865 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-14T07:10:27.867 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:27.867 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:27.870 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:27.867+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:27.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:27.970 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:27.967+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:28.017 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:28.018 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:28.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:28.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:10:28.072 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:28.071+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 142, 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:10:27.016514+0000", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:09:59.535395+0000", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5806080, 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "noscrub", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:10:28.145 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:10:28.146 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:10:28.147 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:10:28.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:10:28.149 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.150 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:10:28.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 5, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 5, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.40000000596046448, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.71628320217132568, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-14T07:10:28.152 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.72000002861022949 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:10:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:10:28.154 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:10:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:10:28.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:10:28.157 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 3.440000057220459, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2.5, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 3.440000057220459, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.5, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.71628320217132568, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.72000002861022949, 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: "error_message": "pool 2 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-14T07:10:28.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:10:28.159 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:10:28.160 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "142", 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 40, 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 142, 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:10:28.161 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:10:28.162 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:10:28.163 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 9.4499998092651367, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.7200000286102295, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 9.4499998092651367, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.7200000286102295, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.71628320217132568, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.86000001430511475, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.72000002861022949, 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: "error_message": "pool 3 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.297698974609375, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 125, 2024-09-14T07:10:28.165 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:10:28.167 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:10:28.169 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 0, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:10:28.170 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 124, 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:10:28.171 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:10:28.175 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 126, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.176 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.177 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:10:28.178 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:10:28.179 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:10:28.180 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:10:28.181 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 132, 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.185 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:10:28.186 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 132, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:10:28.187 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 132, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.193 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:28.196 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:10:28.197 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 65536, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:10:28.198 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:10:28.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:10:28.200 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-09-14T07:10:28.201 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:28.202 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:28.204 INFO:tasks.thrashosds.thrasher:j is {'epoch': 142, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:10:27.016514+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:09:59.535395+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7162832021713257, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7200000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.440000057220459, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 3.440000057220459, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.7162832021713257, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7200000286102295, 'error_message': 'pool 2 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 40, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 9.449999809265137, 'score_stable': 1.7200000286102295, 'optimal_score': 1, 'raw_score_acting': 9.449999809265137, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.7162832021713257, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.7200000286102295, 'error_message': 'pool 3 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.297698974609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 124, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 132, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 132, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 132, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:10:28.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:10:28.205 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:28.203+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:28.212 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:28.212 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:28.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:28.315 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:28.315+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:28.393 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:28.393 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:28.405 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:28.403+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:28.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:28.506 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:28.503+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:28.530 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:10:28.531 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:10:28.547 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-14T07:10:27.247959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1759155113,"num_objects":1389,"num_object_clones":238,"num_object_copies":2778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1389,"num_whiteouts":405,"num_read":301293,"num_read_kb":950301,"num_write":17879,"num_write_kb":304979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":172260973,"num_keys_recovered":413,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14636,"ondisk_log_size":14636,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":312596,"kb_used_data":49772,"kb_used_omap":7,"kb_used_meta":262712,"kb_avail":471546604,"statfs":{"total":483183820800,"available":482863722496,"internally_reserved":0,"allocated":50966528,"data_stored":378132917,"data_compressed":16625350,"data_compressed_allocated":34021376,"data_compressed_original":369401344,"omap_allocated":7973,"internal_metadata":269017307},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,57,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32111238,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":12744,"num_read_kb":41633,"num_write":1166,"num_write_kb":11683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000960"},"pg_stats":[{"pgid":"3.1f","version":"140'207","reported_seq":6168,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.153908+0000","last_change":"2024-09-14T07:09:14.780523+0000","last_active":"2024-09-14T07:10:25.153908+0000","last_peered":"2024-09-14T07:10:25.153908+0000","last_clean":"2024-09-14T07:10:25.153908+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:10:25.153908+0000","last_undegraded":"2024-09-14T07:10:25.153908+0000","last_fullsized":"2024-09-14T07:10:25.153908+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:24:49.687869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018650470000000001,"stat_sum":{"num_bytes":74073616,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5455,"num_read_kb":46490,"num_write":307,"num_write_kb":11115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"137'517","reported_seq":10199,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289045+0000","last_change":"2024-09-14T07:10:12.821162+0000","last_active":"2024-09-14T07:10:25.289045+0000","last_peered":"2024-09-14T07:10:25.289045+0000","last_clean":"2024-09-14T07:10:25.289045+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:10:25.289045+0000","last_undegraded":"2024-09-14T07:10:25.289045+0000","last_fullsized":"2024-09-14T07:10:25.289045+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:36.122243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001909013,"stat_sum":{"num_bytes":70170502,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":18,"num_read":10990,"num_read_kb":46262,"num_write":800,"num_write_kb":14890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"140'1244","reported_seq":15168,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.278595+0000","last_change":"2024-09-14T07:08:33.768407+0000","last_active":"2024-09-14T07:10:25.278595+0000","last_peered":"2024-09-14T07:10:25.278595+0000","last_clean":"2024-09-14T07:10:25.278595+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:10:25.278595+0000","last_undegraded":"2024-09-14T07:10:25.278595+0000","last_fullsized":"2024-09-14T07:10:25.278595+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:08:01.517275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832089999999998,"stat_sum":{"num_bytes":14868778,"num_objects":48,"num_object_clones":4,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":18663,"num_read_kb":30296,"num_write":1235,"num_write_kb":4841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"140'290","reported_seq":18714,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289283+0000","last_change":"2024-09-14T07:08:31.260460+0000","last_active":"2024-09-14T07:10:25.289283+0000","last_peered":"2024-09-14T07:10:25.289283+0000","last_clean":"2024-09-14T07:10:25.289283+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:10:25.289283+0000","last_undegraded":"2024-09-14T07:10:25.289283+0000","last_fullsized":"2024-09-14T07:10:25.289283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:16.820475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030739629999999999,"stat_sum":{"num_bytes":43666484,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":9,"num_read":18254,"num_read_kb":40387,"num_write":305,"num_write_kb":7319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"140'454","reported_seq":8157,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.027034+0000","last_change":"2024-09-14T07:09:14.782995+0000","last_active":"2024-09-14T07:10:25.027034+0000","last_peered":"2024-09-14T07:10:25.027034+0000","last_clean":"2024-09-14T07:10:25.027034+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:10:25.027034+0000","last_undegraded":"2024-09-14T07:10:25.027034+0000","last_fullsized":"2024-09-14T07:10:25.027034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:42:51.148497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020500689999999998,"stat_sum":{"num_bytes":103910254,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":8427,"num_read_kb":21940,"num_write":587,"num_write_kb":10827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"137'312","reported_seq":28273,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.274069+0000","last_change":"2024-09-14T07:10:02.774511+0000","last_active":"2024-09-14T07:10:22.274069+0000","last_peered":"2024-09-14T07:10:22.274069+0000","last_clean":"2024-09-14T07:10:22.274069+0000","last_became_active":"2024-09-14T07:10:02.773571+0000","last_became_peered":"2024-09-14T07:10:02.773571+0000","last_unstale":"2024-09-14T07:10:22.274069+0000","last_undegraded":"2024-09-14T07:10:22.274069+0000","last_fullsized":"2024-09-14T07:10:22.274069+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:33:02.624669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026161399999999999,"stat_sum":{"num_bytes":83192493,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":28552,"num_read_kb":48094,"num_write":366,"num_write_kb":11738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":27032787,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"137'1171","reported_seq":7380,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.251200+0000","last_change":"2024-09-14T07:10:02.775423+0000","last_active":"2024-09-14T07:10:22.251200+0000","last_peered":"2024-09-14T07:10:22.251200+0000","last_clean":"2024-09-14T07:10:22.251200+0000","last_became_active":"2024-09-14T07:10:02.773318+0000","last_became_peered":"2024-09-14T07:10:02.773318+0000","last_unstale":"2024-09-14T07:10:22.251200+0000","last_undegraded":"2024-09-14T07:10:22.251200+0000","last_fullsized":"2024-09-14T07:10:22.251200+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:12.238739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013741249999999999,"stat_sum":{"num_bytes":35154196,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":15,"num_read":11160,"num_read_kb":25592,"num_write":2013,"num_write_kb":9095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":4196551,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"137'329","reported_seq":4357,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.076824+0000","last_change":"2024-09-14T07:09:14.792463+0000","last_active":"2024-09-14T07:10:25.076824+0000","last_peered":"2024-09-14T07:10:25.076824+0000","last_clean":"2024-09-14T07:10:25.076824+0000","last_became_active":"2024-09-14T07:09:14.790767+0000","last_became_peered":"2024-09-14T07:09:14.790767+0000","last_unstale":"2024-09-14T07:10:25.076824+0000","last_undegraded":"2024-09-14T07:10:25.076824+0000","last_fullsized":"2024-09-14T07:10:25.076824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:50:27.664499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017040390000000001,"stat_sum":{"num_bytes":81316040,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":4085,"num_read_kb":11153,"num_write":411,"num_write_kb":14894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"137'204","reported_seq":3334,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040389+0000","last_change":"2024-09-14T07:09:11.408949+0000","last_active":"2024-09-14T07:10:25.040389+0000","last_peered":"2024-09-14T07:10:25.040389+0000","last_clean":"2024-09-14T07:10:25.040389+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:10:25.040389+0000","last_undegraded":"2024-09-14T07:10:25.040389+0000","last_fullsized":"2024-09-14T07:10:25.040389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:50.471206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019945420000000002,"stat_sum":{"num_bytes":41792550,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":6,"num_read":2748,"num_read_kb":19464,"num_write":218,"num_write_kb":4105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"137'523","reported_seq":8669,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.097609+0000","last_change":"2024-09-14T07:07:49.818793+0000","last_active":"2024-09-14T07:10:25.097609+0000","last_peered":"2024-09-14T07:10:25.097609+0000","last_clean":"2024-09-14T07:10:25.097609+0000","last_became_active":"2024-09-14T07:07:47.792297+0000","last_became_peered":"2024-09-14T07:07:47.792297+0000","last_unstale":"2024-09-14T07:10:25.097609+0000","last_undegraded":"2024-09-14T07:10:25.097609+0000","last_fullsized":"2024-09-14T07:10:25.097609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:13:40.089670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767577,"stat_sum":{"num_bytes":50965708,"num_objects":47,"num_object_clones":4,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":9792,"num_read_kb":37347,"num_write":629,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"137'355","reported_seq":4977,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289862+0000","last_change":"2024-09-14T07:09:14.778531+0000","last_active":"2024-09-14T07:10:25.289862+0000","last_peered":"2024-09-14T07:10:25.289862+0000","last_clean":"2024-09-14T07:10:25.289862+0000","last_became_active":"2024-09-14T07:09:14.777794+0000","last_became_peered":"2024-09-14T07:09:14.777794+0000","last_unstale":"2024-09-14T07:10:25.289862+0000","last_undegraded":"2024-09-14T07:10:25.289862+0000","last_fullsized":"2024-09-14T07:10:25.289862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:22.620804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817246,"stat_sum":{"num_bytes":70793265,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":11,"num_read":5902,"num_read_kb":41713,"num_write":419,"num_write_kb":15264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":18194018,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"140'618","reported_seq":10031,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.222896+0000","last_change":"2024-09-14T07:08:31.232572+0000","last_active":"2024-09-14T07:10:25.222896+0000","last_peered":"2024-09-14T07:10:25.222896+0000","last_clean":"2024-09-14T07:10:25.222896+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:10:25.222896+0000","last_undegraded":"2024-09-14T07:10:25.222896+0000","last_fullsized":"2024-09-14T07:10:25.222896+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:11:44.001647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016016470000000001,"stat_sum":{"num_bytes":64520819,"num_objects":55,"num_object_clones":9,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":19,"num_read":11390,"num_read_kb":34963,"num_write":821,"num_write_kb":13696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.005066+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:10:25.005066+0000","last_peered":"2024-09-14T07:10:25.005066+0000","last_clean":"2024-09-14T07:10:25.005066+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:10:25.005066+0000","last_undegraded":"2024-09-14T07:10:25.005066+0000","last_fullsized":"2024-09-14T07:10:25.005066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'515","reported_seq":11141,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289296+0000","last_change":"2024-09-14T07:10:02.776247+0000","last_active":"2024-09-14T07:10:25.289296+0000","last_peered":"2024-09-14T07:10:25.289296+0000","last_clean":"2024-09-14T07:10:25.289296+0000","last_became_active":"2024-09-14T07:10:02.773321+0000","last_became_peered":"2024-09-14T07:10:02.773321+0000","last_unstale":"2024-09-14T07:10:25.289296+0000","last_undegraded":"2024-09-14T07:10:25.289296+0000","last_fullsized":"2024-09-14T07:10:25.289296+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:37:44.099807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002241317,"stat_sum":{"num_bytes":29248512,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":7,"num_read":13002,"num_read_kb":29607,"num_write":618,"num_write_kb":4774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":3295030,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004753+0000","last_change":"2024-09-14T07:10:00.564082+0000","last_active":"2024-09-14T07:10:25.004753+0000","last_peered":"2024-09-14T07:10:25.004753+0000","last_clean":"2024-09-14T07:10:25.004753+0000","last_became_active":"2024-09-14T07:10:00.563259+0000","last_became_peered":"2024-09-14T07:10:00.563259+0000","last_unstale":"2024-09-14T07:10:25.004753+0000","last_undegraded":"2024-09-14T07:10:25.004753+0000","last_fullsized":"2024-09-14T07:10:25.004753+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:00.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"137'432","reported_seq":5056,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.251483+0000","last_change":"2024-09-14T07:10:12.825814+0000","last_active":"2024-09-14T07:10:22.251483+0000","last_peered":"2024-09-14T07:10:22.251483+0000","last_clean":"2024-09-14T07:10:22.251483+0000","last_became_active":"2024-09-14T07:10:12.824726+0000","last_became_peered":"2024-09-14T07:10:12.824726+0000","last_unstale":"2024-09-14T07:10:22.251483+0000","last_undegraded":"2024-09-14T07:10:22.251483+0000","last_fullsized":"2024-09-14T07:10:22.251483+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:03.129989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022044360000000002,"stat_sum":{"num_bytes":37214806,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":12,"num_read":5623,"num_read_kb":16651,"num_write":559,"num_write_kb":4011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":427094,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004679+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:10:25.004679+0000","last_peered":"2024-09-14T07:10:25.004679+0000","last_clean":"2024-09-14T07:10:25.004679+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:10:25.004679+0000","last_undegraded":"2024-09-14T07:10:25.004679+0000","last_fullsized":"2024-09-14T07:10:25.004679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"140'586","reported_seq":5753,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.156008+0000","last_change":"2024-09-14T07:10:12.821383+0000","last_active":"2024-09-14T07:10:25.156008+0000","last_peered":"2024-09-14T07:10:25.156008+0000","last_clean":"2024-09-14T07:10:25.156008+0000","last_became_active":"2024-09-14T07:10:12.820174+0000","last_became_peered":"2024-09-14T07:10:12.820174+0000","last_unstale":"2024-09-14T07:10:25.156008+0000","last_undegraded":"2024-09-14T07:10:25.156008+0000","last_fullsized":"2024-09-14T07:10:25.156008+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:57.411785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426198,"stat_sum":{"num_bytes":11951159,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":7111,"num_read_kb":13701,"num_write":785,"num_write_kb":3655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":492599,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:19.977403+0000","last_change":"2024-09-14T07:10:00.559864+0000","last_active":"2024-09-14T07:10:19.977403+0000","last_peered":"2024-09-14T07:10:19.977403+0000","last_clean":"2024-09-14T07:10:19.977403+0000","last_became_active":"2024-09-14T07:10:00.558149+0000","last_became_peered":"2024-09-14T07:10:00.558149+0000","last_unstale":"2024-09-14T07:10:19.977403+0000","last_undegraded":"2024-09-14T07:10:19.977403+0000","last_fullsized":"2024-09-14T07:10:19.977403+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:27.061139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.5","version":"139'621","reported_seq":3151,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040506+0000","last_change":"2024-09-14T07:07:53.877303+0000","last_active":"2024-09-14T07:10:25.040506+0000","last_peered":"2024-09-14T07:10:25.040506+0000","last_clean":"2024-09-14T07:10:25.040506+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:10:25.040506+0000","last_undegraded":"2024-09-14T07:10:25.040506+0000","last_fullsized":"2024-09-14T07:10:25.040506+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:43:42.932559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029752200000000002,"stat_sum":{"num_bytes":16480557,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":4581,"num_read_kb":17849,"num_write":1006,"num_write_kb":5492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"137'234","reported_seq":8744,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.286227+0000","last_change":"2024-09-14T07:09:11.678793+0000","last_active":"2024-09-14T07:10:22.286227+0000","last_peered":"2024-09-14T07:10:22.286227+0000","last_clean":"2024-09-14T07:10:22.286227+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:10:22.286227+0000","last_undegraded":"2024-09-14T07:10:22.286227+0000","last_fullsized":"2024-09-14T07:10:22.286227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:04:08.829536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001130212,"stat_sum":{"num_bytes":71780352,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":9,"num_read":7885,"num_read_kb":44513,"num_write":307,"num_write_kb":16285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":275,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040134+0000","last_change":"2024-09-14T07:10:00.567828+0000","last_active":"2024-09-14T07:10:25.040134+0000","last_peered":"2024-09-14T07:10:25.040134+0000","last_clean":"2024-09-14T07:10:25.040134+0000","last_became_active":"2024-09-14T07:10:00.562043+0000","last_became_peered":"2024-09-14T07:10:00.562043+0000","last_unstale":"2024-09-14T07:10:25.040134+0000","last_undegraded":"2024-09-14T07:10:25.040134+0000","last_fullsized":"2024-09-14T07:10:25.040134+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:05:49.999748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"137'369","reported_seq":5038,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.138769+0000","last_change":"2024-09-14T07:10:12.826718+0000","last_active":"2024-09-14T07:10:22.138769+0000","last_peered":"2024-09-14T07:10:22.138769+0000","last_clean":"2024-09-14T07:10:22.138769+0000","last_became_active":"2024-09-14T07:10:12.824828+0000","last_became_peered":"2024-09-14T07:10:12.824828+0000","last_unstale":"2024-09-14T07:10:22.138769+0000","last_undegraded":"2024-09-14T07:10:22.138769+0000","last_fullsized":"2024-09-14T07:10:22.138769+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:13:03.893293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018009370000000001,"stat_sum":{"num_bytes":33571207,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":12,"num_read":6650,"num_read_kb":20137,"num_write":297,"num_write_kb":3781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":54,"num_bytes_recovered":1876804,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":246,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.076842+0000","last_change":"2024-09-14T07:10:00.557817+0000","last_active":"2024-09-14T07:10:25.076842+0000","last_peered":"2024-09-14T07:10:25.076842+0000","last_clean":"2024-09-14T07:10:25.076842+0000","last_became_active":"2024-09-14T07:10:00.557388+0000","last_became_peered":"2024-09-14T07:10:00.557388+0000","last_unstale":"2024-09-14T07:10:25.076842+0000","last_undegraded":"2024-09-14T07:10:25.076842+0000","last_fullsized":"2024-09-14T07:10:25.076842+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:12:10.600536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"137'345","reported_seq":11658,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.308883+0000","last_change":"2024-09-14T07:09:14.778597+0000","last_active":"2024-09-14T07:10:22.308883+0000","last_peered":"2024-09-14T07:10:22.308883+0000","last_clean":"2024-09-14T07:10:22.308883+0000","last_became_active":"2024-09-14T07:09:14.777611+0000","last_became_peered":"2024-09-14T07:09:14.777611+0000","last_unstale":"2024-09-14T07:10:22.308883+0000","last_undegraded":"2024-09-14T07:10:22.308883+0000","last_fullsized":"2024-09-14T07:10:22.308883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:57.893492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001859688,"stat_sum":{"num_bytes":22443561,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":14,"num_read":11111,"num_read_kb":13714,"num_write":292,"num_write_kb":6584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003714+0000","last_change":"2024-09-14T07:10:00.561528+0000","last_active":"2024-09-14T07:10:25.003714+0000","last_peered":"2024-09-14T07:10:25.003714+0000","last_clean":"2024-09-14T07:10:25.003714+0000","last_became_active":"2024-09-14T07:10:00.560484+0000","last_became_peered":"2024-09-14T07:10:00.560484+0000","last_unstale":"2024-09-14T07:10:25.003714+0000","last_undegraded":"2024-09-14T07:10:25.003714+0000","last_fullsized":"2024-09-14T07:10:25.003714+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:16:16.642352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"137'277","reported_seq":5742,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.064297+0000","last_change":"2024-09-14T07:10:02.773083+0000","last_active":"2024-09-14T07:10:25.064297+0000","last_peered":"2024-09-14T07:10:25.064297+0000","last_clean":"2024-09-14T07:10:25.064297+0000","last_became_active":"2024-09-14T07:10:02.772053+0000","last_became_peered":"2024-09-14T07:10:02.772053+0000","last_unstale":"2024-09-14T07:10:25.064297+0000","last_undegraded":"2024-09-14T07:10:25.064297+0000","last_fullsized":"2024-09-14T07:10:25.064297+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:15.332776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016406400000000001,"stat_sum":{"num_bytes":54619168,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":5114,"num_read_kb":9947,"num_write":253,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":274464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":284,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040173+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:10:25.040173+0000","last_peered":"2024-09-14T07:10:25.040173+0000","last_clean":"2024-09-14T07:10:25.040173+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:10:25.040173+0000","last_undegraded":"2024-09-14T07:10:25.040173+0000","last_fullsized":"2024-09-14T07:10:25.040173+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":378,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003729+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:10:25.003729+0000","last_peered":"2024-09-14T07:10:25.003729+0000","last_clean":"2024-09-14T07:10:25.003729+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:10:25.003729+0000","last_undegraded":"2024-09-14T07:10:25.003729+0000","last_fullsized":"2024-09-14T07:10:25.003729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"140'360","reported_seq":9131,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289941+0000","last_change":"2024-09-14T07:10:12.824690+0000","last_active":"2024-09-14T07:10:25.289941+0000","last_peered":"2024-09-14T07:10:25.289941+0000","last_clean":"2024-09-14T07:10:25.289941+0000","last_became_active":"2024-09-14T07:10:12.823788+0000","last_became_peered":"2024-09-14T07:10:12.823788+0000","last_unstale":"2024-09-14T07:10:25.289941+0000","last_undegraded":"2024-09-14T07:10:25.289941+0000","last_fullsized":"2024-09-14T07:10:25.289941+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:55.176368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728806,"stat_sum":{"num_bytes":150521428,"num_objects":56,"num_object_clones":9,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":11,"num_read":8632,"num_read_kb":63290,"num_write":498,"num_write_kb":27130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":14496852,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"137'545","reported_seq":6312,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.284469+0000","last_change":"2024-09-14T07:08:33.767747+0000","last_active":"2024-09-14T07:10:22.284469+0000","last_peered":"2024-09-14T07:10:22.284469+0000","last_clean":"2024-09-14T07:10:22.284469+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:10:22.284469+0000","last_undegraded":"2024-09-14T07:10:22.284469+0000","last_fullsized":"2024-09-14T07:10:22.284469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:55.083685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021404409999999999,"stat_sum":{"num_bytes":83919291,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":20,"num_read":7417,"num_read_kb":43172,"num_write":624,"num_write_kb":16762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"138'324","reported_seq":6980,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.064338+0000","last_change":"2024-09-14T07:08:31.134185+0000","last_active":"2024-09-14T07:10:25.064338+0000","last_peered":"2024-09-14T07:10:25.064338+0000","last_clean":"2024-09-14T07:10:25.064338+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:10:25.064338+0000","last_undegraded":"2024-09-14T07:10:25.064338+0000","last_fullsized":"2024-09-14T07:10:25.064338+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:06.304427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001493529,"stat_sum":{"num_bytes":36181008,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":6906,"num_read_kb":26946,"num_write":322,"num_write_kb":3473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"137'879","reported_seq":6947,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-14T07:10:22.297283+0000","last_change":"2024-09-14T07:10:02.776185+0000","last_active":"2024-09-14T07:10:22.297283+0000","last_peered":"2024-09-14T07:10:22.297283+0000","last_clean":"2024-09-14T07:10:22.297283+0000","last_became_active":"2024-09-14T07:10:02.774294+0000","last_became_peered":"2024-09-14T07:10:02.774294+0000","last_unstale":"2024-09-14T07:10:22.297283+0000","last_undegraded":"2024-09-14T07:10:22.297283+0000","last_fullsized":"2024-09-14T07:10:22.297283+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:16.895719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017319410000000001,"stat_sum":{"num_bytes":66279755,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":12,"num_read":9379,"num_read_kb":36398,"num_write":1262,"num_write_kb":13591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":25856356,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.c","version":"140'406","reported_seq":14552,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.040755+0000","last_change":"2024-09-14T07:09:13.779039+0000","last_active":"2024-09-14T07:10:25.040755+0000","last_peered":"2024-09-14T07:10:25.040755+0000","last_clean":"2024-09-14T07:10:25.040755+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:10:25.040755+0000","last_undegraded":"2024-09-14T07:10:25.040755+0000","last_fullsized":"2024-09-14T07:10:25.040755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:57.711682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033078420000000001,"stat_sum":{"num_bytes":55193600,"num_objects":37,"num_object_clones":4,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":15750,"num_read_kb":34800,"num_write":319,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"140'284","reported_seq":3543,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.098660+0000","last_change":"2024-09-14T07:07:53.833921+0000","last_active":"2024-09-14T07:10:25.098660+0000","last_peered":"2024-09-14T07:10:25.098660+0000","last_clean":"2024-09-14T07:10:25.098660+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:10:25.098660+0000","last_undegraded":"2024-09-14T07:10:25.098660+0000","last_fullsized":"2024-09-14T07:10:25.098660+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:43.076004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001157321,"stat_sum":{"num_bytes":32752340,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":3164,"num_read_kb":17058,"num_write":285,"num_write_kb":4854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"140'376","reported_seq":16405,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289760+0000","last_change":"2024-09-14T07:08:31.156808+0000","last_active":"2024-09-14T07:10:25.289760+0000","last_peered":"2024-09-14T07:10:25.289760+0000","last_clean":"2024-09-14T07:10:25.289760+0000","last_became_active":"2024-09-14T07:08:31.110841+0000","last_became_peered":"2024-09-14T07:08:31.110841+0000","last_unstale":"2024-09-14T07:10:25.289760+0000","last_undegraded":"2024-09-14T07:10:25.289760+0000","last_fullsized":"2024-09-14T07:10:25.289760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:32:53.814868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027661420000000001,"stat_sum":{"num_bytes":44772972,"num_objects":34,"num_object_clones":3,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":9,"num_read":17138,"num_read_kb":36275,"num_write":379,"num_write_kb":5774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1082368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"140'367","reported_seq":7654,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.207901+0000","last_change":"2024-09-14T07:07:49.821457+0000","last_active":"2024-09-14T07:10:25.207901+0000","last_peered":"2024-09-14T07:10:25.207901+0000","last_clean":"2024-09-14T07:10:25.207901+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:10:25.207901+0000","last_undegraded":"2024-09-14T07:10:25.207901+0000","last_fullsized":"2024-09-14T07:10:25.207901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:48:10.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003131923,"stat_sum":{"num_bytes":32828057,"num_objects":43,"num_object_clones":5,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":7877,"num_read_kb":13306,"num_write":523,"num_write_kb":16586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"137'259","reported_seq":6884,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.222143+0000","last_change":"2024-09-14T07:09:14.775292+0000","last_active":"2024-09-14T07:10:25.222143+0000","last_peered":"2024-09-14T07:10:25.222143+0000","last_clean":"2024-09-14T07:10:25.222143+0000","last_became_active":"2024-09-14T07:09:14.772671+0000","last_became_peered":"2024-09-14T07:09:14.772671+0000","last_unstale":"2024-09-14T07:10:25.222143+0000","last_undegraded":"2024-09-14T07:10:25.222143+0000","last_fullsized":"2024-09-14T07:10:25.222143+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:58:00.176170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012129109999999999,"stat_sum":{"num_bytes":56167714,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":16,"num_read":6061,"num_read_kb":22656,"num_write":360,"num_write_kb":16646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":83968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"140'448","reported_seq":8573,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.289218+0000","last_change":"2024-09-14T07:08:31.283909+0000","last_active":"2024-09-14T07:10:25.289218+0000","last_peered":"2024-09-14T07:10:25.289218+0000","last_clean":"2024-09-14T07:10:25.289218+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:10:25.289218+0000","last_undegraded":"2024-09-14T07:10:25.289218+0000","last_fullsized":"2024-09-14T07:10:25.289218+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:04:49.483505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459540000000001,"stat_sum":{"num_bytes":60875407,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":9856,"num_read_kb":20615,"num_write":350,"num_write_kb":8566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"140'426","reported_seq":6321,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.004276+0000","last_change":"2024-09-14T07:09:11.813755+0000","last_active":"2024-09-14T07:10:25.004276+0000","last_peered":"2024-09-14T07:10:25.004276+0000","last_clean":"2024-09-14T07:10:25.004276+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:10:25.004276+0000","last_undegraded":"2024-09-14T07:10:25.004276+0000","last_fullsized":"2024-09-14T07:10:25.004276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:13.059416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001615205,"stat_sum":{"num_bytes":59173443,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":20,"num_read":6978,"num_read_kb":21526,"num_write":340,"num_write_kb":7206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"137'284","reported_seq":4333,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T07:10:25.003969+0000","last_change":"2024-09-14T07:10:02.772027+0000","last_active":"2024-09-14T07:10:25.003969+0000","last_peered":"2024-09-14T07:10:25.003969+0000","last_clean":"2024-09-14T07:10:25.003969+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:10:25.003969+0000","last_undegraded":"2024-09-14T07:10:25.003969+0000","last_fullsized":"2024-09-14T07:10:25.003969+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:48:10.697379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001597048,"stat_sum":{"num_bytes":68165684,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":13,"num_read":5574,"num_read_kb":44384,"num_write":286,"num_write_kb":5176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1758564726,"num_objects":1385,"num_object_clones":238,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":405,"num_read":301227,"num_read_kb":950246,"num_write":17776,"num_write_kb":303349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1555,"num_bytes_recovered":171080218,"num_keys_recovered":413,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":48246784,"data_stored":433982763,"data_compressed":18579109,"data_compressed_allocated":38580224,"data_compressed_original":428144128,"omap_allocated":0,"internal_metadata":0},"log_size":14561,"ondisk_log_size":14561,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79424,"kb_used_data":12144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94292416,"statfs":{"total":96636764160,"available":96555433984,"internally_reserved":0,"allocated":12435456,"data_stored":91100714,"data_compressed":4338208,"data_compressed_allocated":8634368,"data_compressed_original":89143296,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,57,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607591,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61724,"kb_used_data":8324,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94310116,"statfs":{"total":96636764160,"available":96573558784,"internally_reserved":0,"allocated":8523776,"data_stored":52390853,"data_compressed":2987999,"data_compressed_allocated":5382144,"data_compressed_original":50787840,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67508,"kb_used_data":9948,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304332,"statfs":{"total":96636764160,"available":96567635968,"internally_reserved":0,"allocated":10186752,"data_stored":77394047,"data_compressed":3289172,"data_compressed_allocated":6742016,"data_compressed_original":75619840,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607592,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62588,"kb_used_data":9192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94309252,"statfs":{"total":96636764160,"available":96572674048,"internally_reserved":0,"allocated":9412608,"data_stored":74217174,"data_compressed":2916331,"data_compressed_allocated":6201344,"data_compressed_original":72620544,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":51539607592,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607592,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41352,"kb_used_data":10164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330488,"statfs":{"total":96636764160,"available":96594419712,"internally_reserved":0,"allocated":10407936,"data_stored":83030129,"data_compressed":3093640,"data_compressed_allocated":7061504,"data_compressed_original":81229824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8728576,"data_stored":81789318,"data_compressed":3102932,"data_compressed_allocated":7041024,"data_compressed_original":80664576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5402624,"data_stored":60083276,"data_compressed":1820382,"data_compressed_allocated":4485120,"data_compressed_original":59631616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7819264,"data_stored":73655659,"data_compressed":2946347,"data_compressed_allocated":6242304,"data_compressed_original":72722944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8560640,"data_stored":76230388,"data_compressed":3329296,"data_compressed_allocated":6758400,"data_compressed_original":75136512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":90379931,"data_compressed":4337914,"data_compressed_allocated":8605696,"data_compressed_original":89085952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6959104,"data_stored":51844191,"data_compressed":3042238,"data_compressed_allocated":5447680,"data_compressed_original":50902528,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:10:28.549 INFO:tasks.thrashosds.thrasher:Setting 3.15 to [3, 2] 2024-09-14T07:10:28.549 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.15', '3', '2'] 2024-09-14T07:10:28.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.15 3 2 2024-09-14T07:10:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:28.572 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:28.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:28.607 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:28.603+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:28.708 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:28.707+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:28.752 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:28.752 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:28.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:28.809 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:28.807+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:28.909 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:28.907+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:28.925 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:28.925 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:28.939 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:29.010 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:29.007+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.15 pg_upmap mapping to [3,2] 2024-09-14T07:10:29.039 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:10:29.075 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:29.075 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:29.090 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:29.111 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:29.111+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:29.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:29.195 INFO:teuthology.orchestra.run.smithi145.stdout:5322 trunc from 0xa25bc00 to 0xc83b000 2024-09-14T07:10:29.212 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:29.211+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:29.218 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:29.218 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:29.234 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:29.267 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:29.268 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:29.281 INFO:teuthology.orchestra.run.smithi145.stdout:5324 trunc from 0xc83b000 to 0x32cf000 2024-09-14T07:10:29.283 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:29.283 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:29.300 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:29.313 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:29.311+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:29.317 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:29.320 INFO:teuthology.orchestra.run.smithi145.stdout:5325 trunc from 0x32cf000 to 0x74fea00 2024-09-14T07:10:29.338 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:29.354 INFO:teuthology.orchestra.run.smithi145.stdout:5326 trunc from 0x74fea00 to 0x2114000 2024-09-14T07:10:29.358 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:29.358 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 177.06650028000001, 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 101.172507096, 2024-09-14T07:10:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 101.17238721699999, 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 101.155162097, 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.482567709999998, 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2675 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1085440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708525843999993, 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.104651067, 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2675 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658478+0000", 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658479+0000", 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2399999999999998e-07 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658482+0000", 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658489+0000", 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8929999999999999e-06 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658503+0000", 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4025e-05 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.762948+0000", 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10444524299999999 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763012+0000", 2024-09-14T07:10:29.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3924000000000007e-05 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763129+0000", 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011765 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708493615999998, 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:29.409 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:29.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:29.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708471600999999, 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:29.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708450666000005, 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.418 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.422 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708429656000007, 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.423 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:29.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.427 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708395956000004, 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:29.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:29.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708375313000005, 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:29.432 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:29.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.436 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708130096000005, 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:29.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:29.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.440 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708098376999999, 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:29.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:29.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708076618000007, 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.708044538999999, 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.449 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:29.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:29.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:29.452 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.706873333000004, 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.453 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.706850369999998, 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.457 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:29.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.706829080000006, 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:29.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:29.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 76.706808856999999, 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:29.466 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:29.470 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:29.472 INFO:teuthology.orchestra.run.smithi145.stdout:5327 write 0xbc6f200 thru 0xbc6fbb5 (0x9b6 bytes) 2024-09-14T07:10:29.472 INFO:teuthology.orchestra.run.smithi145.stdout:5328 trunc from 0xbc6fc00 to 0x51a0a00 2024-09-14T07:10:29.473 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:29.473 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:29.471+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:29.474 INFO:teuthology.orchestra.run.smithi145.stdout:5331 read 0x349800 thru 0x357a4b (0xe24c bytes) 2024-09-14T07:10:29.504 INFO:teuthology.orchestra.run.smithi145.stdout:5334 write 0x6947e00 thru 0x69523fb (0xa5fc bytes) 2024-09-14T07:10:29.514 INFO:teuthology.orchestra.run.smithi145.stdout:5335 trunc from 0x6952400 to 0x2d1de00 2024-09-14T07:10:29.572 INFO:teuthology.orchestra.run.smithi145.stdout:5337 write 0x702ce00 thru 0x7031e14 (0x5015 bytes) 2024-09-14T07:10:29.572 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:29.571+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:29.582 INFO:teuthology.orchestra.run.smithi145.stdout:5338 trunc from 0x7032000 to 0xaacc000 2024-09-14T07:10:29.605 INFO:teuthology.orchestra.run.smithi145.stdout:5339 read 0x6d40e00 thru 0x6d413da (0x5db bytes) 2024-09-14T07:10:29.607 INFO:teuthology.orchestra.run.smithi145.stdout:5340 write 0x1873200 thru 0x1880df7 (0xdbf8 bytes) 2024-09-14T07:10:29.608 INFO:teuthology.orchestra.run.smithi145.stdout:5342 write 0xa3fea00 thru 0xa40345c (0x4a5d bytes) 2024-09-14T07:10:29.621 INFO:teuthology.orchestra.run.smithi145.stdout:5343 trunc from 0xaacc000 to 0xe20de00 2024-09-14T07:10:29.659 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:29.659 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:29.659 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:29.659 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:29.664 INFO:teuthology.orchestra.run.smithi145.stdout:5347 trunc from 0xe20de00 to 0x8ec4400 2024-09-14T07:10:29.673 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:29.671+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:29.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:29.688 INFO:teuthology.orchestra.run.smithi145.stdout:5350 read 0x63e5600 thru 0x63e7c48 (0x2649 bytes) 2024-09-14T07:10:29.690 INFO:teuthology.orchestra.run.smithi145.stdout:5352 write 0x11eb600 thru 0x11f6b6d (0xb56e bytes) 2024-09-14T07:10:29.691 INFO:teuthology.orchestra.run.smithi145.stdout:5353 punch from 0x874e000 to 0x8753c00, (0x5c00 bytes) 2024-09-14T07:10:29.716 INFO:teuthology.orchestra.run.smithi145.stdout:5354 write 0xe55e200 thru 0xe56c8eb (0xe6ec bytes) 2024-09-14T07:10:29.736 INFO:teuthology.orchestra.run.smithi145.stdout:5358 trunc from 0xe56ca00 to 0x2469200 2024-09-14T07:10:29.774 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:29.771+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:29.785 INFO:teuthology.orchestra.run.smithi145.stdout:5359 read 0x9c7400 thru 0x9d6070 (0xec71 bytes) 2024-09-14T07:10:29.849 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:29.849 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:29.849 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:29.849 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:29.849 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:29.863 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:29.875 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:29.875+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:29.976 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:29.975+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 132.72678392899999, 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.025 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:30.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:30.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:30.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:30.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.867308342, 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.031 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.035 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.867206057, 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:30.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.040 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.168154216, 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.683142654999997, 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:30.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:30.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.683108586000003, 2024-09-14T07:10:30.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.053 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.683065240000005, 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.683031717000006, 2024-09-14T07:10:30.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:30.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:30.061 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.682984466999997, 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.065 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.682870218999994, 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:30.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:30.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.069 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.682760524000003, 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.070 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.682694381000005, 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:30.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.447618533, 2024-09-14T07:10:30.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:30.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:30.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.160905097000001, 2024-09-14T07:10:30.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:30.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:10:30.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:10:30.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:10:30.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.544890055, 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:30.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:30.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:10:30.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:10:30.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:10:30.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.096 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.544826356, 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:10:30.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:30.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:30.102 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.544783022000001, 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:10:30.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.544726143, 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:10:30.108 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:10:30.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:10:30.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.544659648, 2024-09-14T07:10:30.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:10:30.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:10:30.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.541775311, 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.125 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.126 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:30.127 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:30.128 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:30.127+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:30.228 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:30.227+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:30.254 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:30.254 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:30.254 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:30.255 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:30.268 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:30.329 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:30.327+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:30.387 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:30.387 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:30.387 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:30.387 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:30.387 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:30.401 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:30.430 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:30.426+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:30.527 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:30.527 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:30.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.69461942500001, 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:30.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:30.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.69384507699999, 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:30.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.68266207900001, 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:30.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:30.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.38573387299999, 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.544 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:30.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.150830049000007, 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.550 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:30.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:30.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:30.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.150268257999997, 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:30.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:30.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.149773690999993, 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:30.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.149156271999999, 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.563 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:30.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:30.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:30.567 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.148728012999996, 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:30.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:30.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:30.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.148332148999998, 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:30.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.147740472999999, 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.576 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:30.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:30.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.144165729999997, 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:30.580 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.584 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.144126377000006, 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:30.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:30.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.144088597000007, 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:30.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.144051894, 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.594 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:30.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.554679960000001, 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:10:30.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:10:30.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.550007527999998, 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:10:30.602 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:10:30.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.549925036000001, 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.607 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:10:30.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:10:30.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.549763915, 2024-09-14T07:10:30.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:30.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.615 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.549587007, 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:10:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:10:30.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:30.620 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:30.622 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:30.619+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:30.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:30.725 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:30.722+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:30.759 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:30.759 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:30.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:30.859 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:30.858+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:30.929 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:30.927+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:30.944 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:30.944 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:30.945 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:30.945 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:30.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:31.047 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:31.046+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:31.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:10:31.132 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:31.127+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:31.135 INFO:teuthology.orchestra.run.smithi145.stdout:5360 write 0xad21a00 thru 0xad236b0 (0x1cb1 bytes) 2024-09-14T07:10:31.138 INFO:teuthology.orchestra.run.smithi145.stdout:5361 write 0x8461600 thru 0x846287c (0x127d bytes) 2024-09-14T07:10:31.138 INFO:teuthology.orchestra.run.smithi145.stdout:5362 write 0x65a7600 thru 0x65a7e5c (0x85d bytes) 2024-09-14T07:10:31.138 INFO:teuthology.orchestra.run.smithi145.stdout:5363 write 0x2c9600 thru 0x2cbcd2 (0x26d3 bytes) 2024-09-14T07:10:31.141 INFO:teuthology.orchestra.run.smithi145.stdout:5364 write 0x2746800 thru 0x274fb1c (0x931d bytes) 2024-09-14T07:10:31.143 INFO:teuthology.orchestra.run.smithi145.stdout:5366 read 0x55d8c00 thru 0x55ded29 (0x612a bytes) 2024-09-14T07:10:31.143 INFO:teuthology.orchestra.run.smithi145.stdout:5367 read 0x5fed600 thru 0x5fee441 (0xe42 bytes) 2024-09-14T07:10:31.143 INFO:teuthology.orchestra.run.smithi145.stdout:5369 write 0xdf6000 thru 0xe00328 (0xa329 bytes) 2024-09-14T07:10:31.145 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.85417867300001, 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.82467104200001, 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.816023683, 2024-09-14T07:10:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 129.14993134299999, 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 128.98320211699999, 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.050201109, 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.294757937, 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.294740015000002, 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.294727245999994, 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.294717070000004, 2024-09-14T07:10:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.294706875000003, 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.294696854999998, 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.241195740999999, 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 43.080888229000003, 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 43.080856689999997, 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 43.080825378, 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 43.080785401, 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 43.071774582000003, 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 43.071165018000002, 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 29.045423219, 2024-09-14T07:10:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:31.244 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:31.245 INFO:teuthology.orchestra.run.smithi145.stdout:5370 trunc from 0xad23800 to 0x4183800 2024-09-14T07:10:31.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:31.247 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:31.242+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:31.346 INFO:teuthology.orchestra.run.smithi145.stdout:5372 read 0x2363000 thru 0x236ab7d (0x7b7e bytes) 2024-09-14T07:10:31.347 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:31.343+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:31.383 INFO:teuthology.orchestra.run.smithi145.stdout:5374 trunc from 0x4183800 to 0x183000 2024-09-14T07:10:31.421 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:31.421 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:31.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:31.439 INFO:teuthology.orchestra.run.smithi145.stdout:5378 punch from 0xa8000 to 0xb2c00, (0xac00 bytes) 2024-09-14T07:10:31.441 INFO:teuthology.orchestra.run.smithi145.stdout:5379 read 0x164600 thru 0x17065a (0xc05b bytes) 2024-09-14T07:10:31.448 INFO:teuthology.orchestra.run.smithi145.stdout:5383 read 0xec200 thru 0xf7844 (0xb645 bytes) 2024-09-14T07:10:31.450 INFO:teuthology.orchestra.run.smithi145.stdout:5385 read 0x47000 thru 0x4db97 (0x6b98 bytes) 2024-09-14T07:10:31.470 INFO:teuthology.orchestra.run.smithi145.stdout:5386 trunc from 0x183000 to 0x7167400 2024-09-14T07:10:31.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:31.487 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:31.486+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:31.513 INFO:teuthology.orchestra.run.smithi145.stdout:5387 trunc from 0x7167400 to 0xcce2a00 2024-09-14T07:10:31.537 INFO:teuthology.orchestra.run.smithi145.stdout:5388 punch from 0x7e4ae00 to 0x7e4e000, (0x3200 bytes) 2024-09-14T07:10:31.541 INFO:teuthology.orchestra.run.smithi145.stdout:5389 read 0x961ea00 thru 0x961f57e (0xb7f bytes) 2024-09-14T07:10:31.541 INFO:teuthology.orchestra.run.smithi145.stdout:5390 read 0xb3cca00 thru 0xb3d7561 (0xab62 bytes) 2024-09-14T07:10:31.541 INFO:teuthology.orchestra.run.smithi145.stdout:5392 write 0x140ac00 thru 0x14151c0 (0xa5c1 bytes) 2024-09-14T07:10:31.541 INFO:teuthology.orchestra.run.smithi145.stdout:5394 punch from 0x32e4e00 to 0x32f2800, (0xda00 bytes) 2024-09-14T07:10:31.541 INFO:teuthology.orchestra.run.smithi145.stdout:5396 write 0x5b7b600 thru 0x5b7e9fa (0x33fb bytes) 2024-09-14T07:10:31.548 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:31.546+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:31.554 INFO:teuthology.orchestra.run.smithi145.stdout:5402 trunc from 0xcce2a00 to 0x46efe00 2024-09-14T07:10:31.556 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:31.556 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:31.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:31.602 INFO:teuthology.orchestra.run.smithi145.stdout:5403 write 0x3d5ca00 thru 0x3d6159f (0x4ba0 bytes) 2024-09-14T07:10:31.633 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:31.644 INFO:teuthology.orchestra.run.smithi145.stdout:5404 write 0x9a25c00 thru 0x9a2d35e (0x775f bytes) 2024-09-14T07:10:31.646 INFO:teuthology.orchestra.run.smithi145.stdout:5406 write 0x3c19200 thru 0x3c1dc02 (0x4a03 bytes) 2024-09-14T07:10:31.649 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:31.647+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:31.650 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:31.653 INFO:teuthology.orchestra.run.smithi145.stdout:5407 trunc from 0x9a2d400 to 0x7a9d400 2024-09-14T07:10:31.668 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:31.690 INFO:teuthology.orchestra.run.smithi145.stdout:5408 read 0x656ce00 thru 0x656ec7b (0x1e7c bytes) 2024-09-14T07:10:31.691 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:31.693 INFO:teuthology.orchestra.run.smithi145.stdout:5410 write 0x47c6c00 thru 0x47d61f5 (0xf5f6 bytes) 2024-09-14T07:10:31.693 INFO:teuthology.orchestra.run.smithi145.stdout:5413 write 0x3456400 thru 0x3462a9d (0xc69e bytes) 2024-09-14T07:10:31.694 INFO:teuthology.orchestra.run.smithi145.stdout:5415 read 0x17c400 thru 0x17e809 (0x240a bytes) 2024-09-14T07:10:31.700 INFO:teuthology.orchestra.run.smithi145.stdout:5417 trunc from 0x7a9d400 to 0x611e400 2024-09-14T07:10:31.717 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:31.717 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:31.737 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:31.737 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:31.738 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:31.738 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:31.738 INFO:teuthology.orchestra.run.smithi145.stdout:5421 clone 39 order 25 su 8388608 sc 12 2024-09-14T07:10:31.749 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:31.747+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:31.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:31.850 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:31.846+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:31.917 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:31.917 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:31.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:31.951 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:31.947+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:32.052 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:32.051+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:32.060 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:10:32.099 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:32.099 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:32.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:32.155 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:32.154+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:32.254 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:32.250+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:32.275 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:32.275 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:32.289 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:32.355 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:32.351+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:32.421 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:32.422 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:32.435 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:32.456 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:32.454+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:32.560 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:32.555+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:32.600 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:32.600 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:32.614 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:32.658 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:32.654+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:32.759 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:32.758+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:32.780 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:32.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:32.860 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:32.859+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:32.918 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:32.919 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:32.933 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:32.961 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:32.958+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:33.061 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:33.058+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:33.106 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:33.106 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:33.123 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:33.137 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone38 from 0x611e400 (overlap 0x183000) to 0x9ac00 2024-09-14T07:10:33.162 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:33.158+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:33.253 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:33.253 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:33.263 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:33.258+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:33.268 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:33.364 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:33.363+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:33.393 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:33.393 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:33.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:33.500 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:33.499+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:33.566 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:33.562+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:33.583 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:33.583 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:33.599 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:33.632 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-14T07:10:33.667 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:33.666+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:33.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:33.761 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:33.762 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:33.768 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:33.767+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:33.775 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:33.776 INFO:teuthology.orchestra.run.smithi145.stdout:5423 trunc from 0x611e400 to 0x8f0800 2024-09-14T07:10:33.869 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:33.867+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:33.911 INFO:teuthology.orchestra.run.smithi145.stdout:5425 read 0x878600 thru 0x8812fa (0x8cfb bytes) 2024-09-14T07:10:33.913 INFO:teuthology.orchestra.run.smithi145.stdout:5426 punch from 0x5f2800 to 0x5f3200, (0xa00 bytes) 2024-09-14T07:10:33.913 INFO:teuthology.orchestra.run.smithi145.stdout:5427 read 0x788400 thru 0x796d7c (0xe97d bytes) 2024-09-14T07:10:33.921 INFO:teuthology.orchestra.run.smithi145.stdout:5429 read 0x5b9400 thru 0x5bb2be (0x1ebf bytes) 2024-09-14T07:10:33.922 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:33.926 INFO:teuthology.orchestra.run.smithi145.stdout:5430 read 0x62a000 thru 0x62f4e0 (0x54e1 bytes) 2024-09-14T07:10:33.930 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:33.930 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:33.930 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:33.931 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:33.932 INFO:teuthology.orchestra.run.smithi145.stdout:5433 read 0x551e00 thru 0x554504 (0x2705 bytes) 2024-09-14T07:10:33.939 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:33.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:33.956 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:33.970 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:33.966+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:33.978 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:33.978 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:33.999 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:33.999 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:34.020 INFO:teuthology.orchestra.run.smithi145.stdout:5434 trunc from 0x8f0800 to 0xc8fe600 2024-09-14T07:10:34.021 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:34.021 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:34.043 INFO:teuthology.orchestra.run.smithi145.stdout:5435 punch from 0x7f93800 to 0x7f9ee00, (0xb600 bytes) 2024-09-14T07:10:34.044 INFO:teuthology.orchestra.run.smithi145.stdout:5436 write 0x36e8800 thru 0x36f7d66 (0xf567 bytes) 2024-09-14T07:10:34.044 INFO:teuthology.orchestra.run.smithi145.stdout:5438 read 0x2f6c800 thru 0x2f714fd (0x4cfe bytes) 2024-09-14T07:10:34.055 INFO:teuthology.orchestra.run.smithi145.stdout:5439 trunc from 0xc8fe600 to 0xd52e200 2024-09-14T07:10:34.064 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:34.064 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:34.064 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T07:10:34.064 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:10:34.064 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:10:34.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T07:10:34.067 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:34.067 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:34.067 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:34.071 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:34.067+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:34.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:34.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:10:34.096 INFO:teuthology.orchestra.run.smithi145.stdout:5440 read 0x9467600 thru 0x946a919 (0x331a bytes) 2024-09-14T07:10:34.098 INFO:teuthology.orchestra.run.smithi145.stdout:5442 punch from 0x9258000 to 0x925ae00, (0x2e00 bytes) 2024-09-14T07:10:34.099 INFO:teuthology.orchestra.run.smithi145.stdout:5443 write 0x7426000 thru 0x742612a (0x12b bytes) 2024-09-14T07:10:34.099 INFO:teuthology.orchestra.run.smithi145.stdout:5445 punch from 0x50e6400 to 0x50e9600, (0x3200 bytes) 2024-09-14T07:10:34.099 INFO:teuthology.orchestra.run.smithi145.stdout:5446 read 0xcd65600 thru 0xcd73c0f (0xe610 bytes) 2024-09-14T07:10:34.101 INFO:teuthology.orchestra.run.smithi145.stdout:5449 read 0x2125c00 thru 0x212ea95 (0x8e96 bytes) 2024-09-14T07:10:34.101 INFO:teuthology.orchestra.run.smithi145.stdout:5450 write 0x8c8f000 thru 0x8c986aa (0x96ab bytes) 2024-09-14T07:10:34.116 INFO:teuthology.orchestra.run.smithi145.stdout:5451 trunc from 0xd52e200 to 0x5db9a00 2024-09-14T07:10:34.163 INFO:teuthology.orchestra.run.smithi145.stdout:5452 read 0x2876400 thru 0x287d082 (0x6c83 bytes) 2024-09-14T07:10:34.166 INFO:teuthology.orchestra.run.smithi145.stdout:5453 read 0x5e2400 thru 0x5ecc2f (0xa830 bytes) 2024-09-14T07:10:34.169 INFO:teuthology.orchestra.run.smithi145.stdout:5454 trunc from 0x5db9a00 to 0x57c9800 2024-09-14T07:10:34.171 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:34.167+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:34.189 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "" 2024-09-14T07:10:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:34.197 INFO:teuthology.orchestra.run.smithi145.stdout:5456 read 0x243a00 thru 0x2453d2 (0x19d3 bytes) 2024-09-14T07:10:34.225 INFO:teuthology.orchestra.run.smithi145.stdout:5457 write 0x60d6e00 thru 0x60d7462 (0x663 bytes) 2024-09-14T07:10:34.231 INFO:teuthology.orchestra.run.smithi145.stdout:5458 write 0x2a19600 thru 0x2a24c30 (0xb631 bytes) 2024-09-14T07:10:34.242 INFO:teuthology.orchestra.run.smithi145.stdout:5460 trunc from 0x60d7600 to 0x2da9800 2024-09-14T07:10:34.252 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 136.962089808, 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 136.932582177, 2024-09-14T07:10:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 136.92393481799999, 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 132.25784247799999, 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 132.09111325200001, 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:34.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:34.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:34.281 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 128.15811224399999, 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:34.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:34.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:34.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.402669071999995, 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.288 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.402651149999997, 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.296 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.402638381000003, 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:34.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:34.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.402628204999999, 2024-09-14T07:10:34.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:34.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:34.305 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.402618009999998, 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:34.310 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.402607990000007, 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:34.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:34.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:34.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.349106875999993, 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:34.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:34.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.188799363999998, 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.188767824999999, 2024-09-14T07:10:34.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:34.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:34.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.188736513000002, 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:34.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:34.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:34.333 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.188696536000002, 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:34.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:34.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.179685716999998, 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:34.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:34.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:34.342 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:34.343 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.179076152999997, 2024-09-14T07:10:34.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:34.343 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:34.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:34.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:34.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.344 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:34.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:34.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:34.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:34.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:34.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:34.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:34.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:34.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:34.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:34.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:34.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:34.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:34.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:34.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.351 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.351 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:34.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:34.352 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.153334354000002, 2024-09-14T07:10:34.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:34.352 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:34.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:34.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:34.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:34.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:34.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:34.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:34.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:34.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:34.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:34.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:34.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:34.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:34.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:34.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.360 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.361 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.361 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:34.362 INFO:teuthology.orchestra.run.smithi145.stdout:5461 punch from 0x23d8600 to 0x23e6c00, (0xe600 bytes) 2024-09-14T07:10:34.362 INFO:teuthology.orchestra.run.smithi145.stdout:5462 read 0x154e200 thru 0x155dba1 (0xf9a2 bytes) 2024-09-14T07:10:34.362 INFO:teuthology.orchestra.run.smithi145.stdout:5465 clone 40 order 20 su 65536 sc 9 2024-09-14T07:10:34.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:34.363 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:34.358+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:34.463 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:34.462+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:34.523 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:34.523 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:34.523 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:34.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:34.579 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:34.578+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:34.665 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:34.662+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:34.703 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:34.703 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:34.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:34.803 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:34.802+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:34.867 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:34.866+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:34.889 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.208010452, 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:34.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:34.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.894 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.817034614999997, 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.895 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:34.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.899 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.817004816999997, 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:34.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816972542000002, 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:34.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:34.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:34.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816914439000001, 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:34.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:34.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816873537000006, 2024-09-14T07:10:34.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:34.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:34.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:34.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:34.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:34.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:34.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:34.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:34.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:34.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:34.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.918 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.918 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.919 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:34.919 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816811728999994, 2024-09-14T07:10:34.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:34.919 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:34.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.920 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:34.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:34.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:34.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:34.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:34.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:34.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:34.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:34.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:34.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:34.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:34.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:34.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:34.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.927 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.928 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.928 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:34.928 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816777412999997, 2024-09-14T07:10:34.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:34.929 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:34.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.930 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:34.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:34.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816758742000005, 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:34.934 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:34.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816734706000005, 2024-09-14T07:10:34.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:34.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:34.942 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816714672000003, 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:34.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816699837000002, 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:34.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816685211000006, 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:34.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:34.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816670709999997, 2024-09-14T07:10:34.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816652954999995, 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.964 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816638061000006, 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:34.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816623949999993, 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:34.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:34.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:34.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:34.972 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816602020999994, 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:34.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:34.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:34.976 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.816580709999997, 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:34.980 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 83.529229809, 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:34.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:34.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:34.985 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:34.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:34.987 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:34.982+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:10:35.087 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:35.087+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:35.164 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:35.165 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:35.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:35.227 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:35.226+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:35.289 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:35.286+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:35.298 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:35.298 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:35.298 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:35.299 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:35.299 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:35.306 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone39 from 0x2da9800 (overlap 0x8f0800) to 0x1c9a00 2024-09-14T07:10:35.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:35.389 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:35.387+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:35.485 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 138.18296998, 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 138.13797967400001, 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.836047218, 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.51471948400001, 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.077591084999995, 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.077580427000001, 2024-09-14T07:10:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.077565413000002, 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.077554497999998, 2024-09-14T07:10:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.077542988000005, 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.077530444999994, 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:35.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:35.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:35.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.548 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070683849000005, 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.553 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070636067999999, 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:35.557 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070622600999997, 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:35.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070611628999998, 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:35.562 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:35.566 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070600866999996, 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:35.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:35.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070584812999996, 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:35.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070568241999993, 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:35.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:35.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070556465999999, 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:35.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:35.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.070545788999993, 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.909369364, 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:35.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:35.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:35.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:10:35.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:35.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:35.595 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:35.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:35.596 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:35.595+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:35.697 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:35.695+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:35.730 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:35.730 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:35.730 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:35.730 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:35.744 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:35.839 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:35.839+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:35.899 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:35.895+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:35.917 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:35.917 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:35.918 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:35.918 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:35.932 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:35.985 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-14T07:10:36.001 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:35.998+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:36.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:36.101 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:36.098+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:36.105 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 183.802433597, 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:36.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:36.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.90844041299999, 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.113 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:36.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:36.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:36.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.908320534, 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:36.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:36.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:36.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:36.127 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.89109541400001, 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:36.132 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.218501027000002, 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:36.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:36.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444426933000003, 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.138 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:36.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:36.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444404918000004, 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:36.142 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:36.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444383982999994, 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:36.150 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444362972999997, 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.154 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444329272999994, 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.155 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:36.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.159 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444308629999995, 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:36.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:36.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444063412999995, 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:36.164 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:36.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:36.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:36.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444031694000003, 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:36.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:36.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.444009934999997, 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:36.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:36.176 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.443977856000004, 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.177 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.442806649999994, 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.181 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.442783687000002, 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.442762396999996, 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.189 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:36.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.442742174000003, 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.194 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:36.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:36.198 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.3150160700000004, 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:36.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:36.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:10:36.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:36.204 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:36.205 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:36.206 INFO:teuthology.orchestra.run.smithi145.stdout:5467 write 0xecbf000 thru 0xecc69db (0x79dc bytes) 2024-09-14T07:10:36.206 INFO:teuthology.orchestra.run.smithi145.stdout:5468 punch from 0x1817400 to 0x181a000, (0x2c00 bytes) 2024-09-14T07:10:36.206 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:36.207 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:36.203+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:36.217 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:36.228 INFO:teuthology.orchestra.run.smithi145.stdout:5471 trunc from 0xecc6a00 to 0xfb3000 2024-09-14T07:10:36.233 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:36.234 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:36.254 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:36.278 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:36.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:36.302 INFO:teuthology.orchestra.run.smithi145.stdout:5472 write 0xe65e00 thru 0xe703aa (0xa5ab bytes) 2024-09-14T07:10:36.306 INFO:teuthology.orchestra.run.smithi145.stdout:5474 read 0x92c800 thru 0x93c2d3 (0xfad4 bytes) 2024-09-14T07:10:36.306 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:36.303+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:36.307 INFO:teuthology.orchestra.run.smithi145.stdout:5475 read 0x633a00 thru 0x6433d2 (0xf9d3 bytes) 2024-09-14T07:10:36.343 INFO:teuthology.orchestra.run.smithi145.stdout:5476 write 0x69d3800 thru 0x69d6b99 (0x339a bytes) 2024-09-14T07:10:36.367 INFO:teuthology.orchestra.run.smithi145.stdout:5479 write 0x715ae00 thru 0x71648e5 (0x9ae6 bytes) 2024-09-14T07:10:36.373 INFO:teuthology.orchestra.run.smithi145.stdout:5480 trunc from 0x7164a00 to 0x93c2600 2024-09-14T07:10:36.393 INFO:teuthology.orchestra.run.smithi145.stdout:5481 write 0x583d400 thru 0x58455b2 (0x81b3 bytes) 2024-09-14T07:10:36.396 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:36.396 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:36.397 INFO:teuthology.orchestra.run.smithi145.stdout:5482 trunc from 0x93c2600 to 0x8755600 2024-09-14T07:10:36.407 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:36.403+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:36.414 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:36.419 INFO:teuthology.orchestra.run.smithi145.stdout:5483 read 0x4134200 thru 0x413fd6f (0xbb70 bytes) 2024-09-14T07:10:36.421 INFO:teuthology.orchestra.run.smithi145.stdout:5484 read 0x92da00 thru 0x935855 (0x7e56 bytes) 2024-09-14T07:10:36.422 INFO:teuthology.orchestra.run.smithi145.stdout:5485 read 0x83f3c00 thru 0x83f508a (0x148b bytes) 2024-09-14T07:10:36.422 INFO:teuthology.orchestra.run.smithi145.stdout:5487 write 0x4a5b600 thru 0x4a691b1 (0xdbb2 bytes) 2024-09-14T07:10:36.426 INFO:teuthology.orchestra.run.smithi145.stdout:5488 trunc from 0x8755600 to 0xa1ce200 2024-09-14T07:10:36.456 INFO:teuthology.orchestra.run.smithi145.stdout:5492 trunc from 0xa1ce200 to 0x9782800 2024-09-14T07:10:36.472 INFO:teuthology.orchestra.run.smithi145.stdout:5495 read 0x53e6e00 thru 0x53eee8d (0x808e bytes) 2024-09-14T07:10:36.473 INFO:teuthology.orchestra.run.smithi145.stdout:5498 read 0x1631200 thru 0x16347cf (0x35d0 bytes) 2024-09-14T07:10:36.473 INFO:teuthology.orchestra.run.smithi145.stdout:5499 punch from 0x53f4e00 to 0x53f5400, (0x600 bytes) 2024-09-14T07:10:36.473 INFO:teuthology.orchestra.run.smithi145.stdout:5500 read 0x704d800 thru 0x704db93 (0x394 bytes) 2024-09-14T07:10:36.474 INFO:teuthology.orchestra.run.smithi145.stdout:5501 punch from 0x3417600 to 0x341a000, (0x2a00 bytes) 2024-09-14T07:10:36.474 INFO:teuthology.orchestra.run.smithi145.stdout:5502 read 0x3febe00 thru 0x3fefc84 (0x3e85 bytes) 2024-09-14T07:10:36.474 INFO:teuthology.orchestra.run.smithi145.stdout:5503 read 0x5d3a00 thru 0x5db091 (0x7692 bytes) 2024-09-14T07:10:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:5505 read 0x3494600 thru 0x3494a75 (0x476 bytes) 2024-09-14T07:10:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:5506 read 0x22fa200 thru 0x22fa989 (0x78a bytes) 2024-09-14T07:10:36.476 INFO:teuthology.orchestra.run.smithi145.stdout:5508 read 0x6b80400 thru 0x6b897e8 (0x93e9 bytes) 2024-09-14T07:10:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:5509 write 0x70fb800 thru 0x710a646 (0xee47 bytes) 2024-09-14T07:10:36.477 INFO:teuthology.orchestra.run.smithi145.stdout:5512 punch from 0x19c7000 to 0x19c8800, (0x1800 bytes) 2024-09-14T07:10:36.484 INFO:teuthology.orchestra.run.smithi145.stdout:5513 trunc from 0x9782800 to 0x6ae1e00 2024-09-14T07:10:36.508 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:36.506+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:36.517 INFO:teuthology.orchestra.run.smithi145.stdout:5518 punch from 0x257e800 to 0x2584600, (0x5e00 bytes) 2024-09-14T07:10:36.541 INFO:teuthology.orchestra.run.smithi145.stdout:5519 write 0x9e1f200 thru 0x9e2ea11 (0xf812 bytes) 2024-09-14T07:10:36.570 INFO:teuthology.orchestra.run.smithi145.stdout:5520 write 0xa2d7c00 thru 0xa2e2c56 (0xb057 bytes) 2024-09-14T07:10:36.573 INFO:teuthology.orchestra.run.smithi145.stdout:5522 read 0x965c000 thru 0x966589c (0x989d bytes) 2024-09-14T07:10:36.573 INFO:teuthology.orchestra.run.smithi145.stdout:5523 read 0x4965600 thru 0x4973d44 (0xe745 bytes) 2024-09-14T07:10:36.573 INFO:teuthology.orchestra.run.smithi145.stdout:5524 punch from 0x79bb400 to 0x79c6c00, (0xb800 bytes) 2024-09-14T07:10:36.593 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:36.593 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:36.601 INFO:teuthology.orchestra.run.smithi145.stdout:5527 write 0xae60a00 thru 0xae6610c (0x570d bytes) 2024-09-14T07:10:36.602 INFO:teuthology.orchestra.run.smithi145.stdout:5528 punch from 0x2bbc400 to 0x2bc9600, (0xd200 bytes) 2024-09-14T07:10:36.603 INFO:teuthology.orchestra.run.smithi145.stdout:5530 write 0xabc9a00 thru 0xabcfe1e (0x641f bytes) 2024-09-14T07:10:36.603 INFO:teuthology.orchestra.run.smithi145.stdout:5531 write 0x54a0800 thru 0x54acad3 (0xc2d4 bytes) 2024-09-14T07:10:36.603 INFO:teuthology.orchestra.run.smithi145.stdout:5532 punch from 0x6ce7e00 to 0x6cf6c00, (0xee00 bytes) 2024-09-14T07:10:36.604 INFO:teuthology.orchestra.run.smithi145.stdout:5533 read 0x3a16200 thru 0x3a25d34 (0xfb35 bytes) 2024-09-14T07:10:36.604 INFO:teuthology.orchestra.run.smithi145.stdout:5534 punch from 0x72b9200 to 0x72bfe00, (0x6c00 bytes) 2024-09-14T07:10:36.604 INFO:teuthology.orchestra.run.smithi145.stdout:5540 write 0x5dc2600 thru 0x5dce2ec (0xbced bytes) 2024-09-14T07:10:36.605 INFO:teuthology.orchestra.run.smithi145.stdout:5541 trunc from 0xae66200 to 0xab9d600 2024-09-14T07:10:36.608 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:36.607+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:36.611 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:36.654 INFO:teuthology.orchestra.run.smithi145.stdout:5542 punch from 0x7136600 to 0x7136a00, (0x400 bytes) 2024-09-14T07:10:36.678 INFO:teuthology.orchestra.run.smithi145.stdout:5548 write 0xe2b4400 thru 0xe2be4bf (0xa0c0 bytes) 2024-09-14T07:10:36.679 INFO:teuthology.orchestra.run.smithi145.stdout:5549 read 0x251aa00 thru 0x2525ae1 (0xb0e2 bytes) 2024-09-14T07:10:36.679 INFO:teuthology.orchestra.run.smithi145.stdout:5551 read 0x8241c00 thru 0x824df0a (0xc30b bytes) 2024-09-14T07:10:36.702 INFO:teuthology.orchestra.run.smithi145.stdout:5552 write 0xe78b000 thru 0xe78b72c (0x72d bytes) 2024-09-14T07:10:36.703 INFO:teuthology.orchestra.run.smithi145.stdout:5554 read 0x25bda00 thru 0x25c9dfb (0xc3fc bytes) 2024-09-14T07:10:36.703 INFO:teuthology.orchestra.run.smithi145.stdout:5556 read 0x1c2ae00 thru 0x1c3a47f (0xf680 bytes) 2024-09-14T07:10:36.704 INFO:teuthology.orchestra.run.smithi145.stdout:5558 write 0xb2e6c00 thru 0xb2e9160 (0x2561 bytes) 2024-09-14T07:10:36.704 INFO:teuthology.orchestra.run.smithi145.stdout:5560 read 0xd2b1000 thru 0xd2bb950 (0xa951 bytes) 2024-09-14T07:10:36.705 INFO:teuthology.orchestra.run.smithi145.stdout:5562 write 0x56a8800 thru 0x56afb81 (0x7382 bytes) 2024-09-14T07:10:36.705 INFO:teuthology.orchestra.run.smithi145.stdout:5563 write 0x1083e00 thru 0x1087bcb (0x3dcc bytes) 2024-09-14T07:10:36.706 INFO:teuthology.orchestra.run.smithi145.stdout:5566 write 0xba0d200 thru 0xba0d77c (0x57d bytes) 2024-09-14T07:10:36.706 INFO:teuthology.orchestra.run.smithi145.stdout:5567 write 0x47db800 thru 0x47eb1bd (0xf9be bytes) 2024-09-14T07:10:36.706 INFO:teuthology.orchestra.run.smithi145.stdout:5568 punch from 0x768e200 to 0x769a800, (0xc600 bytes) 2024-09-14T07:10:36.708 INFO:teuthology.orchestra.run.smithi145.stdout:5575 trunc from 0xe78b800 to 0xecd9000 2024-09-14T07:10:36.708 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:36.706+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:36.741 INFO:teuthology.orchestra.run.smithi145.stdout:5578 read 0x5b41c00 thru 0x5b46021 (0x4422 bytes) 2024-09-14T07:10:36.742 INFO:teuthology.orchestra.run.smithi145.stdout:5583 write 0x16a0800 thru 0x16ae9ea (0xe1eb bytes) 2024-09-14T07:10:36.742 INFO:teuthology.orchestra.run.smithi145.stdout:5584 write 0xe25400 thru 0xe27012 (0x1c13 bytes) 2024-09-14T07:10:36.743 INFO:teuthology.orchestra.run.smithi145.stdout:5588 read 0x236ec00 thru 0x237e01e (0xf41f bytes) 2024-09-14T07:10:36.743 INFO:teuthology.orchestra.run.smithi145.stdout:5590 write 0x7cca00 thru 0x7d7f5b (0xb55c bytes) 2024-09-14T07:10:36.745 INFO:teuthology.orchestra.run.smithi145.stdout:5592 write 0xce9a000 thru 0xcea7069 (0xd06a bytes) 2024-09-14T07:10:36.745 INFO:teuthology.orchestra.run.smithi145.stdout:5593 write 0xebd6200 thru 0xebda118 (0x3f19 bytes) 2024-09-14T07:10:36.745 INFO:teuthology.orchestra.run.smithi145.stdout:5595 write 0xe2f3400 thru 0xe2ffd2e (0xc92f bytes) 2024-09-14T07:10:36.745 INFO:teuthology.orchestra.run.smithi145.stdout:5597 punch from 0xdcb2e00 to 0xdcbd200, (0xa400 bytes) 2024-09-14T07:10:36.746 INFO:teuthology.orchestra.run.smithi145.stdout:5600 read 0x89ecc00 thru 0x89fa377 (0xd778 bytes) 2024-09-14T07:10:36.747 INFO:teuthology.orchestra.run.smithi145.stdout:5602 read 0xa82f000 thru 0xa8318db (0x28dc bytes) 2024-09-14T07:10:36.759 INFO:teuthology.orchestra.run.smithi145.stdout:5603 trunc from 0xecd9000 to 0x7fdcc00 2024-09-14T07:10:36.797 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:36.797 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:36.801 INFO:teuthology.orchestra.run.smithi145.stdout:5604 punch from 0x21aa400 to 0x21b8600, (0xe200 bytes) 2024-09-14T07:10:36.803 INFO:teuthology.orchestra.run.smithi145.stdout:5605 write 0x3918200 thru 0x39255e4 (0xd3e5 bytes) 2024-09-14T07:10:36.803 INFO:teuthology.orchestra.run.smithi145.stdout:5606 read 0x1db8400 thru 0x1dc38a6 (0xb4a7 bytes) 2024-09-14T07:10:36.803 INFO:teuthology.orchestra.run.smithi145.stdout:5607 punch from 0x3a2f600 to 0x3a31600, (0x2000 bytes) 2024-09-14T07:10:36.803 INFO:teuthology.orchestra.run.smithi145.stdout:5608 read 0x2919000 thru 0x29253dc (0xc3dd bytes) 2024-09-14T07:10:36.804 INFO:teuthology.orchestra.run.smithi145.stdout:5609 read 0xbe4400 thru 0xbef96b (0xb56c bytes) 2024-09-14T07:10:36.804 INFO:teuthology.orchestra.run.smithi145.stdout:5610 punch from 0x3bdf400 to 0x3be9e00, (0xaa00 bytes) 2024-09-14T07:10:36.805 INFO:teuthology.orchestra.run.smithi145.stdout:5611 punch from 0x3e89600 to 0x3e93200, (0x9c00 bytes) 2024-09-14T07:10:36.805 INFO:teuthology.orchestra.run.smithi145.stdout:5612 read 0x4741800 thru 0x4748476 (0x6c77 bytes) 2024-09-14T07:10:36.809 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:36.807+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:36.809 INFO:teuthology.orchestra.run.smithi145.stdout:5614 trunc from 0x7fdcc00 to 0x5f47000 2024-09-14T07:10:36.816 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:36.852 INFO:teuthology.orchestra.run.smithi145.stdout:5615 trunc from 0x5f47000 to 0x3156c00 2024-09-14T07:10:36.882 INFO:teuthology.orchestra.run.smithi145.stdout:5617 read 0x2c58400 thru 0x2c65759 (0xd35a bytes) 2024-09-14T07:10:36.893 INFO:teuthology.orchestra.run.smithi145.stdout:5620 trunc from 0x3156c00 to 0x81b8800 2024-09-14T07:10:36.910 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:36.906+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:36.925 INFO:teuthology.orchestra.run.smithi145.stdout:5621 write 0x8986600 thru 0x898fc3d (0x963e bytes) 2024-09-14T07:10:36.926 INFO:teuthology.orchestra.run.smithi145.stdout:5622 punch from 0x2df3c00 to 0x2e00e00, (0xd200 bytes) 2024-09-14T07:10:36.933 INFO:teuthology.orchestra.run.smithi145.stdout:5623 trunc from 0x898fe00 to 0x5559a00 2024-09-14T07:10:36.985 INFO:teuthology.orchestra.run.smithi145.stdout:5624 write 0x5b06a00 thru 0x5b08758 (0x1d59 bytes) 2024-09-14T07:10:36.987 INFO:teuthology.orchestra.run.smithi145.stdout:5625 write 0x28f5c00 thru 0x28f71ea (0x15eb bytes) 2024-09-14T07:10:36.988 INFO:teuthology.orchestra.run.smithi145.stdout:5627 read 0x42d4000 thru 0x42d5724 (0x1725 bytes) 2024-09-14T07:10:36.988 INFO:teuthology.orchestra.run.smithi145.stdout:5628 punch from 0x56600 to 0x5d600, (0x7000 bytes) 2024-09-14T07:10:36.989 INFO:teuthology.orchestra.run.smithi145.stdout:5629 punch from 0x1a12600 to 0x1a20200, (0xdc00 bytes) 2024-09-14T07:10:36.989 INFO:teuthology.orchestra.run.smithi145.stdout:5630 punch from 0x5559e00 to 0x555e600, (0x4800 bytes) 2024-09-14T07:10:36.989 INFO:teuthology.orchestra.run.smithi145.stdout:5632 read 0x54a1a00 thru 0x54b199f (0xffa0 bytes) 2024-09-14T07:10:36.990 INFO:teuthology.orchestra.run.smithi145.stdout:5633 read 0x2b23e00 thru 0x2b25711 (0x1912 bytes) 2024-09-14T07:10:36.990 INFO:teuthology.orchestra.run.smithi145.stdout:5634 read 0x3aede00 thru 0x3af67b4 (0x89b5 bytes) 2024-09-14T07:10:36.991 INFO:teuthology.orchestra.run.smithi145.stdout:5636 read 0x4bde00 thru 0x4cc788 (0xe989 bytes) 2024-09-14T07:10:36.995 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:36.995 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:36.997 INFO:teuthology.orchestra.run.smithi145.stdout:5638 write 0x569d000 thru 0x56ac457 (0xf458 bytes) 2024-09-14T07:10:36.997 INFO:teuthology.orchestra.run.smithi145.stdout:5639 read 0x1a65600 thru 0x1a6ec75 (0x9676 bytes) 2024-09-14T07:10:36.997 INFO:teuthology.orchestra.run.smithi145.stdout:5641 punch from 0x3987600 to 0x3992e00, (0xb800 bytes) 2024-09-14T07:10:37.011 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:37.010+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:37.016 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:37.028 INFO:teuthology.orchestra.run.smithi145.stdout:5643 write 0xce4f600 thru 0xce54f1a (0x591b bytes) 2024-09-14T07:10:37.029 INFO:teuthology.orchestra.run.smithi145.stdout:5644 punch from 0xb1a5200 to 0xb1ac400, (0x7200 bytes) 2024-09-14T07:10:37.035 INFO:teuthology.orchestra.run.smithi145.stdout:5645 trunc from 0xce55000 to 0x9584000 2024-09-14T07:10:37.058 INFO:teuthology.orchestra.run.smithi145.stdout:5646 read 0x8f93200 thru 0x8f963c6 (0x31c7 bytes) 2024-09-14T07:10:37.060 INFO:teuthology.orchestra.run.smithi145.stdout:5647 punch from 0x51f6e00 to 0x51fb200, (0x4400 bytes) 2024-09-14T07:10:37.060 INFO:teuthology.orchestra.run.smithi145.stdout:5649 flatten 2024-09-14T07:10:37.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:10:37.112 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:37.110+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:37.116 INFO:teuthology.orchestra.run.smithi145.stdout:5655 write 0x45d3200 thru 0x45de3c8 (0xb1c9 bytes) 2024-09-14T07:10:37.116 INFO:teuthology.orchestra.run.smithi145.stdout:5656 read 0x5b76400 thru 0x5b80a79 (0xa67a bytes) 2024-09-14T07:10:37.117 INFO:teuthology.orchestra.run.smithi145.stdout:5657 write 0x92a4200 thru 0x92ae09e (0x9e9f bytes) 2024-09-14T07:10:37.124 INFO:teuthology.orchestra.run.smithi145.stdout:5659 trunc from 0x9584000 to 0x6e39e00 2024-09-14T07:10:37.155 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:37.156 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:37.163 INFO:teuthology.orchestra.run.smithi145.stdout:5661 read 0x55a5a00 thru 0x55a8877 (0x2e78 bytes) 2024-09-14T07:10:37.165 INFO:teuthology.orchestra.run.smithi145.stdout:5662 read 0x1887000 thru 0x1889d89 (0x2d8a bytes) 2024-09-14T07:10:37.165 INFO:teuthology.orchestra.run.smithi145.stdout:5663 read 0x5cba600 thru 0x5cc3770 (0x9171 bytes) 2024-09-14T07:10:37.166 INFO:teuthology.orchestra.run.smithi145.stdout:5666 read 0x55a5200 thru 0x55a96fd (0x44fe bytes) 2024-09-14T07:10:37.166 INFO:teuthology.orchestra.run.smithi145.stdout:5671 write 0x32d6400 thru 0x32dd216 (0x6e17 bytes) 2024-09-14T07:10:37.167 INFO:teuthology.orchestra.run.smithi145.stdout:5672 clone 41 order 22 su 131072 sc 5 2024-09-14T07:10:37.171 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:37.215 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:37.215+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:37.288 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:37.288 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:37.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:37.344 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:37.343+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:37.414 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:37.410+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:37.421 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:37.421 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:37.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:37.515 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:37.511+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:37.611 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:37.611 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:37.615 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:37.615+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:37.627 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:37.717 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:37.714+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:37.754 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:37.754 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:37.768 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:37.818 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:37.815+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:37.919 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:37.918+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:37.939 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:37.939 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:37.954 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:38.021 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:38.019+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:10:38.118 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:38.118 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:38.121 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:38.118+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:38.134 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:38.222 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:38.218+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:38.291 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:38.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:38.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:38.323 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:38.319+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:38.424 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:38.422+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:38.471 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:38.471 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:38.488 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:38.502 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:38.502 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:38.519 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:38.519 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:38.524 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:38.522+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:38.536 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:38.557 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:38.557 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:38.577 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:38.598 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:38.620 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:38.621 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:38.626 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:38.622+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:38.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:38.727 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:38.723+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:38.810 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 141.507155936, 2024-09-14T07:10:38.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.812 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:38.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:38.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 141.46216562999999, 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.824 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:38.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:38.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:38.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.16023317400001, 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:38.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:38.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:38.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:38.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 136.83890543999999, 2024-09-14T07:10:38.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.843 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:38.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:38.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:38.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:38.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:38.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.401777041000003, 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:38.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:38.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.401766382999995, 2024-09-14T07:10:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:38.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:38.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:38.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.869 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.401751368999996, 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:38.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:38.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:38.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:38.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.401740454000006, 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:38.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.401728943999998, 2024-09-14T07:10:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:38.885 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:38.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:38.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:38.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:38.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:38.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:38.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.897 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.901 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.401716401000002, 2024-09-14T07:10:38.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:38.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:38.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:38.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:38.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394869804999999, 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.915 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:38.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:38.923 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394822024000007, 2024-09-14T07:10:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394808557000005, 2024-09-14T07:10:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:38.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:38.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.939 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394797585000006, 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:38.940 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:38.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394786823000004, 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:38.947 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:38.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394770769000004, 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:38.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.958 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394754198000001, 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:38.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:38.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.963 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394742421999993, 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.964 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:38.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:38.969 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 92.394731745000001, 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:38.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.233555320000001, 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:10:38.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:38.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:10:38.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:38.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:38.983 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:38.983 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:38.984 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:38.985 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:38.983+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:39.072 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-14T07:10:39.084 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:39.082+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:39.121 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:39.121 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:39.121 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:39.121 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:39.135 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:39.179 INFO:teuthology.orchestra.run.smithi145.stdout:5673 trunc from 0x6e39e00 to 0x2e01200 2024-09-14T07:10:39.186 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:39.182+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:39.206 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:39.206 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:39.206 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T07:10:39.206 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.526474 2024-09-14T07:10:39.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.5264743247885844 2024-09-14T07:10:39.263 INFO:teuthology.orchestra.run.smithi145.stdout:5675 write 0x7179a00 thru 0x717a647 (0xc48 bytes) 2024-09-14T07:10:39.266 INFO:teuthology.orchestra.run.smithi145.stdout:5676 read 0x3a61400 thru 0x3a662ae (0x4eaf bytes) 2024-09-14T07:10:39.267 INFO:teuthology.orchestra.run.smithi145.stdout:5678 punch from 0x5ac6600 to 0x5ac8e00, (0x2800 bytes) 2024-09-14T07:10:39.268 INFO:teuthology.orchestra.run.smithi145.stdout:5680 read 0x64d7e00 thru 0x64e50c2 (0xd2c3 bytes) 2024-09-14T07:10:39.268 INFO:teuthology.orchestra.run.smithi145.stdout:5682 write 0x2788400 thru 0x279733d (0xef3e bytes) 2024-09-14T07:10:39.269 INFO:teuthology.orchestra.run.smithi145.stdout:5683 read 0x2953a00 thru 0x2960944 (0xcf45 bytes) 2024-09-14T07:10:39.271 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:39.271 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:39.271 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:39.271 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:39.271 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:39.287 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:39.287 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:39.282+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:39.335 INFO:teuthology.orchestra.run.smithi145.stdout:5685 write 0xbc73000 thru 0xbc812af (0xe2b0 bytes) 2024-09-14T07:10:39.337 INFO:teuthology.orchestra.run.smithi145.stdout:5688 read 0x67b4e00 thru 0x67b8b9b (0x3d9c bytes) 2024-09-14T07:10:39.337 INFO:teuthology.orchestra.run.smithi145.stdout:5690 write 0x5a35000 thru 0x5a38c1f (0x3c20 bytes) 2024-09-14T07:10:39.338 INFO:teuthology.orchestra.run.smithi145.stdout:5691 flatten 2024-09-14T07:10:39.388 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:39.387+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:39.393 INFO:teuthology.orchestra.run.smithi145.stdout:5693 write 0x48e7a00 thru 0x48e8716 (0xd17 bytes) 2024-09-14T07:10:39.394 INFO:teuthology.orchestra.run.smithi145.stdout:5694 read 0x6a25000 thru 0x6a2a0b4 (0x50b5 bytes) 2024-09-14T07:10:39.394 INFO:teuthology.orchestra.run.smithi145.stdout:5695 read 0xb345e00 thru 0xb35233a (0xc53b bytes) 2024-09-14T07:10:39.395 INFO:teuthology.orchestra.run.smithi145.stdout:5699 write 0x18d8600 thru 0x18e5e1f (0xd820 bytes) 2024-09-14T07:10:39.395 INFO:teuthology.orchestra.run.smithi145.stdout:5701 write 0x41bba00 thru 0x41cb6a1 (0xfca2 bytes) 2024-09-14T07:10:39.396 INFO:teuthology.orchestra.run.smithi145.stdout:5702 read 0x51f8a00 thru 0x5206548 (0xdb49 bytes) 2024-09-14T07:10:39.397 INFO:teuthology.orchestra.run.smithi145.stdout:5705 read 0x1ae2200 thru 0x1ae25ab (0x3ac bytes) 2024-09-14T07:10:39.397 INFO:teuthology.orchestra.run.smithi145.stdout:5706 punch from 0xae16400 to 0xae19600, (0x3200 bytes) 2024-09-14T07:10:39.398 INFO:teuthology.orchestra.run.smithi145.stdout:5707 read 0x240c600 thru 0x240f8e9 (0x32ea bytes) 2024-09-14T07:10:39.398 INFO:teuthology.orchestra.run.smithi145.stdout:5708 read 0x274b400 thru 0x2751259 (0x5e5a bytes) 2024-09-14T07:10:39.399 INFO:teuthology.orchestra.run.smithi145.stdout:5709 write 0x440e400 thru 0x4416823 (0x8424 bytes) 2024-09-14T07:10:39.399 INFO:teuthology.orchestra.run.smithi145.stdout:5710 read 0x129e800 thru 0x12a9656 (0xae57 bytes) 2024-09-14T07:10:39.407 INFO:teuthology.orchestra.run.smithi145.stdout:5711 trunc from 0xbc81400 to 0xce80600 2024-09-14T07:10:39.429 INFO:teuthology.orchestra.run.smithi145.stdout:5712 write 0x6602e00 thru 0x660d540 (0xa741 bytes) 2024-09-14T07:10:39.431 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:39.431 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 187.12970012700001, 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:39.432 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:39.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.438 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 111.235706943, 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:39.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:39.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:39.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.446 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 111.235587064, 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:39.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:39.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:39.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:39.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:39.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:39.454 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 111.21836194399999, 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:39.459 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 88.545767557000005, 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:39.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:39.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771693463000005, 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.465 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.469 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771671448000006, 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.470 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771650512999997, 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:39.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771629503, 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:39.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771595802999997, 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.483 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:39.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771575159999998, 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.488 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:39.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:39.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771329942999998, 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.492 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.496 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771298224000006, 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:39.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:39.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:39.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771276465, 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.771244386000006, 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:39.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.770073179999997, 2024-09-14T07:10:39.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:39.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:39.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.770050217000005, 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:39.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:39.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:39.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.770028926999998, 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:39.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:39.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.770008704000006, 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:39.522 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:39.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:39.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.6422825999999997, 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:39.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:10:39.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:39.532 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:39.535 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5713 write 0xe549000 thru 0xe54d85a (0x485b bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5715 punch from 0x22a1c00 to 0x22ace00, (0xb200 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5717 read 0xadbb800 thru 0xadbdbc4 (0x23c5 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5718 punch from 0xe0c9200 to 0xe0cb000, (0x1e00 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5720 write 0xa93de00 thru 0xa948a63 (0xac64 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5723 punch from 0x7b30400 to 0x7b3d400, (0xd000 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5726 write 0x32dcc00 thru 0x32e9cf3 (0xd0f4 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5727 write 0xccace00 thru 0xccb2f36 (0x6137 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5729 punch from 0x5b46000 to 0x5b54200, (0xe200 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5730 trunc from 0xe54da00 to 0xbc94e00 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5731 read 0x1a15400 thru 0x1a17b37 (0x2738 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5734 punch from 0x2821400 to 0x282fc00, (0xe800 bytes) 2024-09-14T07:10:39.536 INFO:teuthology.orchestra.run.smithi145.stdout:5735 read 0x4295200 thru 0x429b156 (0x5f57 bytes) 2024-09-14T07:10:39.537 INFO:teuthology.orchestra.run.smithi145.stdout:5736 punch from 0x5965200 to 0x5966400, (0x1200 bytes) 2024-09-14T07:10:39.537 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:39.530+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:39.544 INFO:teuthology.orchestra.run.smithi145.stdout:5737 trunc from 0xbc94e00 to 0x7ccd200 2024-09-14T07:10:39.580 INFO:teuthology.orchestra.run.smithi145.stdout:5741 trunc from 0x7ccd200 to 0x535e000 2024-09-14T07:10:39.635 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:39.630+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:39.643 INFO:teuthology.orchestra.run.smithi145.stdout:5743 write 0xe3da800 thru 0xe3e7ea4 (0xd6a5 bytes) 2024-09-14T07:10:39.644 INFO:teuthology.orchestra.run.smithi145.stdout:5744 write 0xcd22200 thru 0xcd2a86a (0x866b bytes) 2024-09-14T07:10:39.647 INFO:teuthology.orchestra.run.smithi145.stdout:5748 trunc from 0xe3e8000 to 0xce66800 2024-09-14T07:10:39.707 INFO:teuthology.orchestra.run.smithi145.stdout:5749 write 0xde10e00 thru 0xde1ff28 (0xf129 bytes) 2024-09-14T07:10:39.708 INFO:teuthology.orchestra.run.smithi145.stdout:5755 read 0x4de5200 thru 0x4de6a4d (0x184e bytes) 2024-09-14T07:10:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:39.722 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:39.735 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:39.736 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:39.731+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:39.736 INFO:teuthology.orchestra.run.smithi145.stdout:5757 trunc from 0xde20000 to 0x2a22e00 2024-09-14T07:10:39.807 INFO:teuthology.orchestra.run.smithi145.stdout:5758 write 0x872fa00 thru 0x8734660 (0x4c61 bytes) 2024-09-14T07:10:39.836 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:39.835+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:39.841 INFO:teuthology.orchestra.run.smithi145.stdout:5761 trunc from 0x8734800 to 0x1828000 2024-09-14T07:10:39.881 INFO:teuthology.orchestra.run.smithi145.stdout:5763 trunc from 0x1828000 to 0x275ae00 2024-09-14T07:10:39.921 INFO:teuthology.orchestra.run.smithi145.stdout:5764 trunc from 0x275ae00 to 0x932b200 2024-09-14T07:10:39.922 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:39.922 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:39.922 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:39.922 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:39.922 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:39.936 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:39.935+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:39.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:39.940 INFO:teuthology.orchestra.run.smithi145.stdout:5765 punch from 0x921c400 to 0x9229400, (0xd000 bytes) 2024-09-14T07:10:39.941 INFO:teuthology.orchestra.run.smithi145.stdout:5767 write 0x684e000 thru 0x685cdca (0xedcb bytes) 2024-09-14T07:10:39.941 INFO:teuthology.orchestra.run.smithi145.stdout:5771 punch from 0x47bac00 to 0x47bb000, (0x400 bytes) 2024-09-14T07:10:39.964 INFO:teuthology.orchestra.run.smithi145.stdout:5772 trunc from 0x932b200 to 0x652400 2024-09-14T07:10:39.994 INFO:teuthology.orchestra.run.smithi145.stdout:5773 read 0xa8c00 thru 0xab458 (0x2859 bytes) 2024-09-14T07:10:40.034 INFO:teuthology.orchestra.run.smithi145.stdout:5774 write 0x88d7e00 thru 0x88e16f2 (0x98f3 bytes) 2024-09-14T07:10:40.035 INFO:teuthology.orchestra.run.smithi145.stdout:5775 read 0x6ca5800 thru 0x6cad843 (0x8044 bytes) 2024-09-14T07:10:40.038 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:40.034+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:40.075 INFO:teuthology.orchestra.run.smithi145.stdout:5776 write 0xd8f1200 thru 0xd8f5839 (0x463a bytes) 2024-09-14T07:10:40.076 INFO:teuthology.orchestra.run.smithi145.stdout:5777 read 0x1a2e200 thru 0x1a319f0 (0x37f1 bytes) 2024-09-14T07:10:40.077 INFO:teuthology.orchestra.run.smithi145.stdout:5778 read 0x8f66e00 thru 0x8f6c217 (0x5418 bytes) 2024-09-14T07:10:40.077 INFO:teuthology.orchestra.run.smithi145.stdout:5781 write 0x2ead400 thru 0x2ebb8ad (0xe4ae bytes) 2024-09-14T07:10:40.077 INFO:teuthology.orchestra.run.smithi145.stdout:5782 read 0xf66400 thru 0xf69a4d (0x364e bytes) 2024-09-14T07:10:40.078 INFO:teuthology.orchestra.run.smithi145.stdout:5784 read 0x8757e00 thru 0x8760264 (0x8465 bytes) 2024-09-14T07:10:40.078 INFO:teuthology.orchestra.run.smithi145.stdout:5785 punch from 0xd260200 to 0xd26d200, (0xd000 bytes) 2024-09-14T07:10:40.078 INFO:teuthology.orchestra.run.smithi145.stdout:5786 write 0x8033200 thru 0x8034e33 (0x1c34 bytes) 2024-09-14T07:10:40.086 INFO:teuthology.orchestra.run.smithi145.stdout:5789 trunc from 0xd8f5a00 to 0x94aae00 2024-09-14T07:10:40.113 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:40.113 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:40.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.814439177, 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:40.114 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:40.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:40.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:40.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 129.95496359000001, 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.120 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:40.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:40.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:40.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:40.128 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 129.95486130500001, 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:40.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:40.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.133 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 129.25580946400001, 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.134 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:40.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:40.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:40.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770797903000002, 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:40.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:40.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770763833999993, 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.144 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:40.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770720487999995, 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.148 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:40.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770686964999996, 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.152 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:40.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770639715000002, 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.156 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:40.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770525466999999, 2024-09-14T07:10:40.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:40.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.164 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770415772000007, 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:40.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.770349628999995, 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.169 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:40.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.535273781000001, 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.248560345000001, 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.178 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:10:40.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:10:40.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.184 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.632545303000001, 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:40.185 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:40.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.632481603999999, 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.191 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:10:40.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.632438270000002, 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:40.197 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.202 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.632381390999999, 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:40.203 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:10:40.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:10:40.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.632314896, 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:10:40.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:10:40.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:10:40.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.629430558999999, 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:10:40.214 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:10:40.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:10:40.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:10:40.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:10:40.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.220 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.220 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.220 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:40.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:10:40.223 INFO:teuthology.orchestra.run.smithi145.stdout:5791 read 0x3fb4200 thru 0x3fbadd2 (0x6bd3 bytes) 2024-09-14T07:10:40.223 INFO:teuthology.orchestra.run.smithi145.stdout:5793 write 0x5ef4e00 thru 0x5ef61de (0x13df bytes) 2024-09-14T07:10:40.223 INFO:teuthology.orchestra.run.smithi145.stdout:5794 read 0xd5fc00 thru 0xd69910 (0x9d11 bytes) 2024-09-14T07:10:40.223 INFO:teuthology.orchestra.run.smithi145.stdout:5797 write 0xd256000 thru 0xd265d4a (0xfd4b bytes) 2024-09-14T07:10:40.223 INFO:teuthology.orchestra.run.smithi145.stdout:5800 trunc from 0xd265e00 to 0x79c8200 2024-09-14T07:10:40.224 INFO:teuthology.orchestra.run.smithi145.stdout:5801 trunc from 0x79c8200 to 0x5518200 2024-09-14T07:10:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.0 primary-affinity to 0.526474 (8345032) 2024-09-14T07:10:40.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:40.224 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:40.219+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:40.242 INFO:teuthology.orchestra.run.smithi145.stdout:5802 read 0x98e400 thru 0x996dc1 (0x89c2 bytes) 2024-09-14T07:10:40.244 INFO:teuthology.orchestra.run.smithi145.stdout:5805 read 0x357600 thru 0x35f802 (0x8203 bytes) 2024-09-14T07:10:40.257 INFO:teuthology.orchestra.run.smithi145.stdout:5809 trunc from 0x5518200 to 0x9ae8400 2024-09-14T07:10:40.274 INFO:teuthology.orchestra.run.smithi145.stdout:5810 write 0x19a8800 thru 0x19b44dd (0xbcde bytes) 2024-09-14T07:10:40.282 INFO:teuthology.orchestra.run.smithi145.stdout:5811 trunc from 0x9ae8400 to 0x7d83800 2024-09-14T07:10:40.322 INFO:teuthology.orchestra.run.smithi145.stdout:5812 trunc from 0x7d83800 to 0x1104000 2024-09-14T07:10:40.323 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:40.319+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:40.353 INFO:teuthology.orchestra.run.smithi145.stdout:5814 trunc from 0x1104000 to 0x178e00 2024-09-14T07:10:40.408 INFO:teuthology.orchestra.run.smithi145.stdout:5815 write 0xa464400 thru 0xa471750 (0xd351 bytes) 2024-09-14T07:10:40.409 INFO:teuthology.orchestra.run.smithi145.stdout:5817 read 0x83ea800 thru 0x83f0946 (0x6147 bytes) 2024-09-14T07:10:40.409 INFO:teuthology.orchestra.run.smithi145.stdout:5819 write 0x816600 thru 0x824408 (0xde09 bytes) 2024-09-14T07:10:40.418 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:40.419 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:40.424 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:40.422+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:40.435 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:40.440 INFO:teuthology.orchestra.run.smithi145.stdout:5820 write 0xce4f800 thru 0xce55349 (0x5b4a bytes) 2024-09-14T07:10:40.442 INFO:teuthology.orchestra.run.smithi145.stdout:5824 punch from 0x40c2e00 to 0x40cbc00, (0x8e00 bytes) 2024-09-14T07:10:40.443 INFO:teuthology.orchestra.run.smithi145.stdout:5826 read 0x939ea00 thru 0x93abbb6 (0xd1b7 bytes) 2024-09-14T07:10:40.444 INFO:teuthology.orchestra.run.smithi145.stdout:5829 write 0x39d8600 thru 0x39dc118 (0x3b19 bytes) 2024-09-14T07:10:40.444 INFO:teuthology.orchestra.run.smithi145.stdout:5831 read 0x99a9e00 thru 0x99ad329 (0x352a bytes) 2024-09-14T07:10:40.444 INFO:teuthology.orchestra.run.smithi145.stdout:5832 punch from 0x4866200 to 0x4869200, (0x3000 bytes) 2024-09-14T07:10:40.444 INFO:teuthology.orchestra.run.smithi145.stdout:5837 read 0x520f600 thru 0x521309a (0x3a9b bytes) 2024-09-14T07:10:40.445 INFO:teuthology.orchestra.run.smithi145.stdout:5839 read 0x3d7a600 thru 0x3d7d703 (0x3104 bytes) 2024-09-14T07:10:40.445 INFO:teuthology.orchestra.run.smithi145.stdout:5841 read 0x39c2600 thru 0x39d0d3c (0xe73d bytes) 2024-09-14T07:10:40.445 INFO:teuthology.orchestra.run.smithi145.stdout:5842 write 0xaec8a00 thru 0xaed80fa (0xf6fb bytes) 2024-09-14T07:10:40.445 INFO:teuthology.orchestra.run.smithi145.stdout:5845 write 0x26fe400 thru 0x26fe84d (0x44e bytes) 2024-09-14T07:10:40.445 INFO:teuthology.orchestra.run.smithi145.stdout:5846 read 0x9b52200 thru 0x9b5aaa2 (0x88a3 bytes) 2024-09-14T07:10:40.449 INFO:teuthology.orchestra.run.smithi145.stdout:5847 trunc from 0xce55400 to 0xb3ed000 2024-09-14T07:10:40.477 INFO:teuthology.orchestra.run.smithi145.stdout:5849 write 0x4b0de00 thru 0x4b186f9 (0xa8fa bytes) 2024-09-14T07:10:40.479 INFO:teuthology.orchestra.run.smithi145.stdout:5851 write 0x5f9000 thru 0x608f63 (0xff64 bytes) 2024-09-14T07:10:40.480 INFO:teuthology.orchestra.run.smithi145.stdout:5852 read 0x72ba000 thru 0x72c7161 (0xd162 bytes) 2024-09-14T07:10:40.480 INFO:teuthology.orchestra.run.smithi145.stdout:5855 punch from 0x9ae3600 to 0x9ae9c00, (0x6600 bytes) 2024-09-14T07:10:40.481 INFO:teuthology.orchestra.run.smithi145.stdout:5856 punch from 0x41eb600 to 0x41ed800, (0x2200 bytes) 2024-09-14T07:10:40.481 INFO:teuthology.orchestra.run.smithi145.stdout:5857 write 0x73f2800 thru 0x74019bb (0xf1bc bytes) 2024-09-14T07:10:40.482 INFO:teuthology.orchestra.run.smithi145.stdout:5858 punch from 0x705400 to 0x70ec00, (0x9800 bytes) 2024-09-14T07:10:40.482 INFO:teuthology.orchestra.run.smithi145.stdout:5860 clone 42 order 21 su 131072 sc 10 2024-09-14T07:10:40.525 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:40.523+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:40.606 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:40.607 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:40.607 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:40.607 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:40.607 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:40.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:40.620 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:40.664 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:40.663+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:40.727 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:40.723+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:40.757 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:40.758 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:40.774 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:40.787 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:40.787 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:40.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:40.787 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:40.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 167.95461640299999, 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.788 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:40.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 167.953842055, 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:40.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:40.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 138.94265905699999, 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.798 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:40.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 138.645730851, 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:40.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:40.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:40.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:40.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.809 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.410827026999996, 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.810 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:40.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.410265236000001, 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:40.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:40.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.409770668999997, 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:40.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:40.823 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.409153250000003, 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:40.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:40.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.408724991, 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:40.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.408329127000002, 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:40.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:40.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.407737451000003, 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:40.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.404162708000001, 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:40.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.404123354999996, 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:40.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.404085574999996, 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:40.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:40.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:40.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 94.404048872000004, 2024-09-14T07:10:40.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.859 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.814676937999998, 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:10:40.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.810004505999999, 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.864 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:10:40.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.809922014000001, 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.809760893000004, 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:10:40.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:10:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.809583985000003, 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:10:40.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:10:40.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:40.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:40.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:40.883 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:40.884 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:40.885 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:40.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:40.887 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:40.888 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:40.882+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:40.986 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:40.983+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:41.062 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:41.062 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:41.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:41.086 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:41.083+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:10:41.187 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:41.187+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:41.210 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:41.210 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:41.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:41.289 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:41.287+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:41.351 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:41.351 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:41.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:41.389 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:41.387+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:41.490 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:41.486+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:41.493 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:41.493 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:41.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:41.591 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:41.587+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:41.680 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:41.680 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:41.692 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:41.690+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:41.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:41.793 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:41.791+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:41.863 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:41.864 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:41.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:41.894 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:41.891+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:41.995 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:41.991+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:42.004 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:42.004 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:42.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:42.095 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:42.094+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:42.150 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:42.150 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:42.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:42.207 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:42.206+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:42.297 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:42.294+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:42.334 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:42.334 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:42.348 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:42.398 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:42.394+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:42.481 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:42.481 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:42.496 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:42.498 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:42.494+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:42.599 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:42.594+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:42.623 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:42.623 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:42.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:42.724 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:42.722+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:42.801 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:42.798+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:42.806 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:42.806 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:42.820 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:42.829 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-14T07:10:42.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:42.935 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:42.934+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:43.002 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:42.998+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:43.008 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:43.008 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:43.025 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:43.059 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:43.060 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:43.076 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:43.093 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:43.094 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:43.103 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:43.098+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:43.114 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:43.114 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:43.135 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:43.135 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:43.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:10:43.154 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:43.160 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:43.160 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:43.160 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:43.160 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:43.160 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:43.176 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:43.215 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:43.214+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:43.305 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:43.302+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:43.355 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:43.355 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:43.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 146.05577119200001, 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:43.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:43.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.19629560499999, 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:43.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:43.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:43.365 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.19619331999999, 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:43.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 132.49714147899999, 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.012129917999999, 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.374 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:43.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.012095849000005, 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.012052503000007, 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:43.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:43.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:43.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.385 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.012018979999993, 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.011971729999999, 2024-09-14T07:10:43.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:43.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.011857481999996, 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:43.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.011747787000004, 2024-09-14T07:10:43.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.011681644000006, 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.401 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:43.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:43.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 40.776605795999998, 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:43.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:10:43.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 31.489892359999999, 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.873877318000002, 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:43.416 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.873813619, 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.873770284999999, 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.427 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:10:43.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:10:43.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:10:43.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.873713406, 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:10:43.433 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:10:43.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.873646911000002, 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:43.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:10:43.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.870762574, 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.445 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:43.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.450 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.451 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.451 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:43.452 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:43.452 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:43.450+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:43.554 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:43.550+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:43.559 INFO:teuthology.orchestra.run.smithi145.stdout:5861 write 0x365f400 thru 0x3669af7 (0xa6f8 bytes) 2024-09-14T07:10:43.561 INFO:teuthology.orchestra.run.smithi145.stdout:5862 punch from 0x5091e00 to 0x509bc00, (0x9e00 bytes) 2024-09-14T07:10:43.592 INFO:teuthology.orchestra.run.smithi145.stdout:5863 read 0x9bd6000 thru 0x9bd8b26 (0x2b27 bytes) 2024-09-14T07:10:43.592 INFO:teuthology.orchestra.run.smithi145.stdout:5864 read 0x843bc00 thru 0x844016e (0x456f bytes) 2024-09-14T07:10:43.592 INFO:teuthology.orchestra.run.smithi145.stdout:5865 read 0x3173800 thru 0x317b719 (0x7f1a bytes) 2024-09-14T07:10:43.593 INFO:teuthology.orchestra.run.smithi145.stdout:5866 write 0x3acc600 thru 0x3ad001c (0x3a1d bytes) 2024-09-14T07:10:43.593 INFO:teuthology.orchestra.run.smithi145.stdout:5867 punch from 0x7ade800 to 0x7ae2800, (0x4000 bytes) 2024-09-14T07:10:43.612 INFO:teuthology.orchestra.run.smithi145.stdout:5870 punch from 0x7571800 to 0x7572600, (0xe00 bytes) 2024-09-14T07:10:43.619 INFO:teuthology.orchestra.run.smithi145.stdout:5876 read 0x85e2200 thru 0x85e3fff (0x1e00 bytes) 2024-09-14T07:10:43.619 INFO:teuthology.orchestra.run.smithi145.stdout:5877 read 0x3ef9a00 thru 0x3f079cd (0xdfce bytes) 2024-09-14T07:10:43.629 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:43.629 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:43.632 INFO:teuthology.orchestra.run.smithi145.stdout:5878 trunc from 0xb3ed000 to 0xd36b600 2024-09-14T07:10:43.645 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:43.653 INFO:teuthology.orchestra.run.smithi145.stdout:5881 write 0x3052600 thru 0x305f221 (0xcc22 bytes) 2024-09-14T07:10:43.656 INFO:teuthology.orchestra.run.smithi145.stdout:5882 read 0x7242000 thru 0x72490f5 (0x70f6 bytes) 2024-09-14T07:10:43.656 INFO:teuthology.orchestra.run.smithi145.stdout:5883 write 0x85c800 thru 0x86238f (0x5b90 bytes) 2024-09-14T07:10:43.658 INFO:teuthology.orchestra.run.smithi145.stdout:5884 read 0x2f38800 thru 0x2f3d8b0 (0x50b1 bytes) 2024-09-14T07:10:43.658 INFO:teuthology.orchestra.run.smithi145.stdout:5885 read 0x50e7200 thru 0x50f6527 (0xf328 bytes) 2024-09-14T07:10:43.659 INFO:teuthology.orchestra.run.smithi145.stdout:5887 write 0x56ff800 thru 0x5702b7f (0x3380 bytes) 2024-09-14T07:10:43.691 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:43.690+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:43.708 INFO:teuthology.orchestra.run.smithi145.stdout:5888 write 0xeb08a00 thru 0xeb0905e (0x65f bytes) 2024-09-14T07:10:43.710 INFO:teuthology.orchestra.run.smithi145.stdout:5893 write 0xd261c00 thru 0xd271650 (0xfa51 bytes) 2024-09-14T07:10:43.710 INFO:teuthology.orchestra.run.smithi145.stdout:5894 punch from 0x54cc000 to 0x54cee00, (0x2e00 bytes) 2024-09-14T07:10:43.711 INFO:teuthology.orchestra.run.smithi145.stdout:5895 punch from 0xd4a1c00 to 0xd4acc00, (0xb000 bytes) 2024-09-14T07:10:43.711 INFO:teuthology.orchestra.run.smithi145.stdout:5897 read 0xc3aea00 thru 0xc3b6207 (0x7808 bytes) 2024-09-14T07:10:43.712 INFO:teuthology.orchestra.run.smithi145.stdout:5898 read 0x7088600 thru 0x708b2f4 (0x2cf5 bytes) 2024-09-14T07:10:43.712 INFO:teuthology.orchestra.run.smithi145.stdout:5902 clone 43 order 19 su 131072 sc 6 2024-09-14T07:10:43.754 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:43.750+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:43.774 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:43.774 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:43.774 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:43.774 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:43.774 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:43.788 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:43.876 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:43.874+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:43.957 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:43.954+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:43.957 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:43.957 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:43.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:43.957 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:43.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.957 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:43.957 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 171.12377341300001, 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:43.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:43.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 171.12299906499999, 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:43.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:43.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.11181606700001, 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:43.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:43.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.975 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.81488786099999, 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:43.976 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:43.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:43.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.579984037000003, 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:43.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.579422246000007, 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.987 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:43.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.578927679000003, 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:43.992 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:43.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:43.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:43.997 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.578310259999995, 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:43.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:43.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.577882001000006, 2024-09-14T07:10:44.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:44.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:44.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.577486136999994, 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:44.007 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:44.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.576894460999995, 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.012 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.573319717999993, 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:44.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.573280365000002, 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:44.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.573242585000003, 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.027 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:44.032 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.573205881999996, 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:44.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.983833947999997, 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.037 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:10:44.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:10:44.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.042 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.979161515999998, 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.043 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.045 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.047 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.979079024000001, 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.048 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:10:44.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:44.052 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.978917903000003, 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:10:44.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:10:44.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:44.057 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 35.978740995000003, 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:44.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:10:44.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:44.062 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:44.064 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:44.062+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:44.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:10:44.164 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:44.162+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:44.204 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:44.204 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:44.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:44.265 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:44.262+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:44.304 INFO:teuthology.orchestra.run.smithi145.stdout:truncating image image_client.0-clone42 from 0xeb09200 (overlap 0xb3ed000) to 0x59f6800 2024-09-14T07:10:44.344 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:44.344 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:44.344 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:44.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:44.403 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:44.402+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:44.467 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:44.466+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:44.537 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.24611477400001, 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.216607143, 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.207959784, 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 142.54186744399999, 2024-09-14T07:10:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 142.37513821799999, 2024-09-14T07:10:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 138.44213721, 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.686694037999999, 2024-09-14T07:10:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.686676116000001, 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.686663347000007, 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.686653171000003, 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.686642976000002, 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.686632955999997, 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 87.633131841999997, 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.472824330000002, 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.472792791000003, 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.472761478999999, 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.472721501999999, 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.463710683000002, 2024-09-14T07:10:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.463101119000001, 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.437359319999999, 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:44.623 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:44.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:44.624 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:44.622+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:44.724 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:44.722+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:44.801 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:44.801 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:44.801 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:44.801 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:44.816 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:44.825 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:44.822+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:44.926 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:44.922+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:44.938 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-14T07:10:44.989 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:44.989 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:44.989 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:44.989 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:44.989 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:45.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:45.026 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:45.022+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:45.127 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:45.126+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:45.129 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.44686434600001, 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055888508999999, 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055858710999999, 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055826436000004, 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055768333000003, 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055727430999994, 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055665622999996, 2024-09-14T07:10:45.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055631306999999, 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:45.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:45.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055612636000006, 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:45.162 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:45.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:45.166 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055588599999993, 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:45.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055568566000005, 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.174 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055553731000003, 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055539104999994, 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:45.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055524603999999, 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:45.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:45.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055506848999997, 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:45.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055491954999994, 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:45.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:45.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055477843999995, 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:45.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055455914999996, 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 98.055434603999998, 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:45.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:45.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:45.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 93.768083703000002, 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:45.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:45.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:45.212 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:45.212 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:45.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:45.225 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:45.226 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:45.226 INFO:tasks.thrashosds.thrasher:Reweighting osd 5 to 0.5415863332365612 2024-09-14T07:10:45.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 0.5415863332365612 2024-09-14T07:10:45.227 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:45.226+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:45.328 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:45.326+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:45.390 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:45.390 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:45.390 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:45.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:45.414 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:45.428 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:45.426+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:45.446 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:45.446 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:45.463 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:45.464 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:45.486 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:45.509 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:45.510 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:45.529 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:45.526+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:45.537 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:45.537 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:45.546 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:45.547 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:45.554 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:45.651 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:45.650+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:45.720 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:45.720 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:45.731 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:45.726+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:45.734 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:45.832 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:45.830+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:45.881 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:45.881 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:45.896 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:45.940 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:45.938+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:46.021 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:46.022 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:46.033 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:46.030+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:46.036 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:46.134 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:46.130+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 0.541586 (8aa5) 2024-09-14T07:10:46.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:10:46.203 INFO:teuthology.orchestra.run.smithi145.stdout:5903 trunc from 0xeb09200 to 0x78cde00 2024-09-14T07:10:46.209 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:46.209 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:46.223 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:46.268 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:46.266+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:46.335 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:46.330+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:46.344 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:46.344 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:46.360 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:46.448 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:46.446+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:46.527 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:46.527 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:46.535 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:46.534+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:46.541 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:46.637 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:46.634+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:46.709 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:46.709 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:46.723 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:46.737 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:46.734+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:46.838 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:46.834+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:46.884 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:46.885 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:46.898 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:46.940 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:46.938+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:47.040 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:47.038+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:47.064 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:47.064 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:47.078 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:47.163 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:10:47.167 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:47.166+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:47.241 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:47.238+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:47.252 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:47.252 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:47.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:47.342 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:47.338+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:47.397 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:47.397 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:47.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:47.443 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:47.438+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:47.544 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:47.542+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:47.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:47.589 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:47.589 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:47.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:47.651 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:47.650+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:47.709 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:47.727 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:47.727 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:47.745 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:47.745 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:47.746 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:47.742+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:47.766 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:47.766 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:47.772 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 150.48267242, 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:47.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:47.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:47.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 150.453164789, 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:47.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 150.44451742999999, 2024-09-14T07:10:47.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:47.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:47.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.790 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 145.77842509000001, 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:47.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 145.61169586400001, 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:47.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:47.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 141.67869485599999, 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:47.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:47.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.923251683999993, 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:47.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:47.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.923233761999995, 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:47.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.923220993000001, 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:47.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.923210816999998, 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:47.822 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:47.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:47.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:47.827 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.923200621999996, 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:47.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.923190602000005, 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:47.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.869689488000006, 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.838 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:47.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.709381976000003, 2024-09-14T07:10:47.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:47.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:47.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:47.848 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.709350436999998, 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:47.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:47.853 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.709319125, 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.709279148, 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:47.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.700268328999996, 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:47.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.868 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.699658765000002, 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:47.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:47.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:47.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:47.873 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.673916966, 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:47.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:47.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:47.879 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:47.879 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:47.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:47.881 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:47.881 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:47.883 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:47.878+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:47.981 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:47.978+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:48.054 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:48.055 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:48.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:48.082 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:48.078+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:48.183 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:48.182+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:48.269 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:48.269 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:48.269 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:48.269 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:48.269 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:48.274 INFO:teuthology.orchestra.run.smithi145.stdout:5904 trunc from 0x78cde00 to 0x7d2e00 2024-09-14T07:10:48.284 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:48.282+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:48.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:48.375 INFO:teuthology.orchestra.run.smithi145.stdout:5909 write 0x91fac00 thru 0x920129e (0x669f bytes) 2024-09-14T07:10:48.376 INFO:teuthology.orchestra.run.smithi145.stdout:5910 read 0x77a800 thru 0x77d506 (0x2d07 bytes) 2024-09-14T07:10:48.385 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:48.382+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:48.387 INFO:teuthology.orchestra.run.smithi145.stdout:5912 trunc from 0x9201400 to 0xda22e00 2024-09-14T07:10:48.414 INFO:teuthology.orchestra.run.smithi145.stdout:5913 trunc from 0xda22e00 to 0xb5a2a00 2024-09-14T07:10:48.431 INFO:teuthology.orchestra.run.smithi145.stdout:5914 write 0x4433e00 thru 0x4436285 (0x2486 bytes) 2024-09-14T07:10:48.436 INFO:teuthology.orchestra.run.smithi145.stdout:5917 trunc from 0xb5a2a00 to 0x9dbc400 2024-09-14T07:10:48.459 INFO:teuthology.orchestra.run.smithi145.stdout:5920 trunc from 0x9dbc400 to 0xbeab000 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 146.78657557599999, 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:48.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:48.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.39559973900001, 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.472 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:48.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:48.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.475 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.39556994100001, 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:48.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395537666, 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:48.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:48.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.39547956299999, 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:48.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:48.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395438661, 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:48.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:48.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395376853, 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:48.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:48.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395342537, 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:48.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:48.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395323866, 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:48.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.39529983, 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395279796, 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395264961, 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395250335, 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395235834, 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395218079, 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395203185, 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395189074, 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395167145, 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.395145834, 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.107794932999994, 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:48.549 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:48.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:48.551 INFO:teuthology.orchestra.run.smithi145.stdout:5921 trunc from 0xbeab000 to 0x11eb200 2024-09-14T07:10:48.551 INFO:teuthology.orchestra.run.smithi145.stdout:5922 read 0xaf5400 thru 0xafc7ff (0x7400 bytes) 2024-09-14T07:10:48.551 INFO:teuthology.orchestra.run.smithi145.stdout:5923 trunc from 0x11eb200 to 0xcf75800 2024-09-14T07:10:48.552 INFO:teuthology.orchestra.run.smithi145.stdout:5924 read 0xbb46000 thru 0xbb4c86d (0x686e bytes) 2024-09-14T07:10:48.552 INFO:teuthology.orchestra.run.smithi145.stdout:5928 write 0x1f0b600 thru 0x1f0b8c9 (0x2ca bytes) 2024-09-14T07:10:48.552 INFO:teuthology.orchestra.run.smithi145.stdout:5929 read 0xabfa00 thru 0xac7b83 (0x8184 bytes) 2024-09-14T07:10:48.552 INFO:teuthology.orchestra.run.smithi145.stdout:5931 read 0xb632c00 thru 0xb638abc (0x5ebd bytes) 2024-09-14T07:10:48.552 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:48.546+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:48.562 INFO:teuthology.orchestra.run.smithi145.stdout:5933 trunc from 0xcf75800 to 0x3b3d400 2024-09-14T07:10:48.584 INFO:teuthology.orchestra.run.smithi145.stdout:5935 read 0x1178800 thru 0x117a044 (0x1845 bytes) 2024-09-14T07:10:48.587 INFO:teuthology.orchestra.run.smithi145.stdout:5936 read 0x2305c00 thru 0x2313e1e (0xe21f bytes) 2024-09-14T07:10:48.587 INFO:teuthology.orchestra.run.smithi145.stdout:5946 read 0x379ca00 thru 0x37ac745 (0xfd46 bytes) 2024-09-14T07:10:48.587 INFO:teuthology.orchestra.run.smithi145.stdout:5947 punch from 0x2f3a200 to 0x2f41200, (0x7000 bytes) 2024-09-14T07:10:48.620 INFO:teuthology.orchestra.run.smithi145.stdout:5951 write 0xbe72800 thru 0xbe763f2 (0x3bf3 bytes) 2024-09-14T07:10:48.635 INFO:teuthology.orchestra.run.smithi145.stdout:5953 trunc from 0xbe76400 to 0x4819e00 2024-09-14T07:10:48.651 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:48.646+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:48.675 INFO:teuthology.orchestra.run.smithi145.stdout:5957 read 0x30f200 thru 0x313dc1 (0x4bc2 bytes) 2024-09-14T07:10:48.678 INFO:teuthology.orchestra.run.smithi145.stdout:5958 read 0x41fca00 thru 0x4205501 (0x8b02 bytes) 2024-09-14T07:10:48.695 INFO:teuthology.orchestra.run.smithi145.stdout:5960 trunc from 0x4819e00 to 0xaf92e00 2024-09-14T07:10:48.714 INFO:teuthology.orchestra.run.smithi145.stdout:5961 read 0x5a3ea00 thru 0x5a496fc (0xacfd bytes) 2024-09-14T07:10:48.717 INFO:teuthology.orchestra.run.smithi145.stdout:5962 punch from 0x1dd7000 to 0x1de3800, (0xc800 bytes) 2024-09-14T07:10:48.741 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:48.741 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:48.741 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:48.741 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:48.745 INFO:teuthology.orchestra.run.smithi145.stdout:5964 write 0xea47800 thru 0xea57407 (0xfc08 bytes) 2024-09-14T07:10:48.746 INFO:teuthology.orchestra.run.smithi145.stdout:5966 flatten 2024-09-14T07:10:48.751 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:48.750+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:48.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:48.801 INFO:teuthology.orchestra.run.smithi145.stdout:5968 read 0x9e27800 thru 0x9e2896d (0x116e bytes) 2024-09-14T07:10:48.814 INFO:teuthology.orchestra.run.smithi145.stdout:5969 trunc from 0xea57600 to 0x839dc00 2024-09-14T07:10:48.845 INFO:teuthology.orchestra.run.smithi145.stdout:5970 punch from 0x5cf4a00 to 0x5cfdc00, (0x9200 bytes) 2024-09-14T07:10:48.848 INFO:teuthology.orchestra.run.smithi145.stdout:5972 read 0x690e800 thru 0x69186d7 (0x9ed8 bytes) 2024-09-14T07:10:48.852 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:48.850+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:48.892 INFO:teuthology.orchestra.run.smithi145.stdout:5980 write 0xcaa9e00 thru 0xcab486a (0xaa6b bytes) 2024-09-14T07:10:48.893 INFO:teuthology.orchestra.run.smithi145.stdout:5984 write 0xbc33200 thru 0xbc3a6ac (0x74ad bytes) 2024-09-14T07:10:48.893 INFO:teuthology.orchestra.run.smithi145.stdout:5985 clone 44 order 24 su 524288 sc 9 2024-09-14T07:10:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:48.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:48.953 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:48.950+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:49.054 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:49.050+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:49.071 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.76881970599999, 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:49.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:49.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:49.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.7238294, 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.078 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:49.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:49.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:49.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.083 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.421896944, 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:49.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:49.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:49.089 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.10056921, 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:49.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.663440811, 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.66343015299999, 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.100 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:49.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:49.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.66341513899999, 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:49.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.663404224, 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:49.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.663392714, 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:49.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.663380171, 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:49.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:49.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:49.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.656533575, 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:49.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.126 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.126 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:49.126 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.65648579400001, 2024-09-14T07:10:49.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:49.126 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.130 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.656472327, 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.656461355, 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.656450593, 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:49.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:49.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:49.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.656434539, 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.148 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.656417968, 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:49.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.65640619200001, 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.656395515, 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:49.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.495219089999999, 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:49.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:49.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:49.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:49.170 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:49.171 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:49.166+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:49.188 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:10:49.270 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:49.266+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:49.303 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:49.303 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:49.304 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:49.304 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:49.318 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:49.371 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:49.370+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:49.472 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:49.470+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:49.486 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:49.486 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:49.486 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:49.486 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:49.486 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:49.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:49.572 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:49.570+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:49.621 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:49.621 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:49.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 197.31816231600001, 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:49.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 121.424169132, 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 121.42404925300001, 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 121.406824133, 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 98.734229745999997, 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.960155651999997, 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.960133636999998, 2024-09-14T07:10:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.960112702000004, 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.960091692000006, 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.960057992000003, 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.960037349000004, 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.959792132000004, 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.959760412999998, 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.959738654000006, 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.959706574999998, 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.958535369000003, 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.958512405999997, 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.958491116000005, 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.958470892999998, 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.830744789000001, 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:10:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:10:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.722 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:49.723 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:49.723 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:49.722+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:49.823 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:49.822+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:49.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22085 3.e 3.76977bae (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e157)", 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:49.250143+0000", 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.64010739100000003, 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.64013063100000001, 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "waiting for peered", 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:49.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22085 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250143+0000", 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250143+0000", 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250145+0000", 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1620000000000001e-06 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:49.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250150+0000", 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7950000000000004e-06 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250164+0000", 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3743999999999999e-05 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250171+0000", 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8770000000000004e-06 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:49.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250268+0000", 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6810000000000003e-05 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:49.250282+0000", 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4209999999999999e-05 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:10:49.894 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:49.904 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:49.931 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:49.930+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:50.024 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:50.022+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:50.058 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:50.073 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:50.073 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:50.073 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:50.073 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:50.073 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:50.075 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:50.087 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:50.091 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:50.109 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:50.110 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:50.125 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:50.122+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:50.127 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:50.127 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:50.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:50.144 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:50.181 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:10:50.226 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:50.222+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:50.230 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:50.230 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:50.245 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:50.332 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:50.330+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:50.413 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:50.413 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:50.427 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:50.426+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:50.428 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:50.529 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:50.526+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:50.594 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:50.594 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:50.609 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:50.652 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:50.650+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:50.730 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:50.726+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:50.781 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:50.781 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:50.783 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-14T07:10:50.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:50.831 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:50.826+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:50.931 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:50.930+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:50.937 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:50.937 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:50.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:51.032 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:51.030+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:51.086 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:51.087 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:51.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:51.147 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:51.146+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:51.171 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:51.172 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:51.172 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-14T07:10:51.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-14T07:10:51.232 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:51.263 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:51.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:51.265 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:51.262+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:51.365 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:51.362+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:51.429 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:51.429 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:51.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:51.487 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:51.486+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:51.509 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-09-14T07:10:51.506+0000 7f1e6b537640 -1 mon.a@0(leader).osd e158 definitely_dead 0 2024-09-14T07:10:51.567 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:51.566+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:51.606 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:51.606 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:51.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:51.668 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:51.666+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:51.698 INFO:teuthology.orchestra.run.smithi145.stdout:5987 trunc from 0xcab4a00 to 0xcde7a00 2024-09-14T07:10:51.733 INFO:teuthology.orchestra.run.smithi145.stdout:5989 read 0x5c68e00 thru 0x5c7057a (0x777b bytes) 2024-09-14T07:10:51.736 INFO:teuthology.orchestra.run.smithi145.stdout:5990 write 0xaa34e00 thru 0xaa41c15 (0xce16 bytes) 2024-09-14T07:10:51.737 INFO:teuthology.orchestra.run.smithi145.stdout:5991 read 0x4194800 thru 0x4195d1a (0x151b bytes) 2024-09-14T07:10:51.737 INFO:teuthology.orchestra.run.smithi145.stdout:5993 read 0xc4a2800 thru 0xc4afce1 (0xd4e2 bytes) 2024-09-14T07:10:51.737 INFO:teuthology.orchestra.run.smithi145.stdout:5994 punch from 0xccb4000 to 0xccbe000, (0xa000 bytes) 2024-09-14T07:10:51.738 INFO:teuthology.orchestra.run.smithi145.stdout:5995 read 0x6cc1a00 thru 0x6cc66b4 (0x4cb5 bytes) 2024-09-14T07:10:51.738 INFO:teuthology.orchestra.run.smithi145.stdout:5996 write 0x81c4e00 thru 0x81d4d65 (0xff66 bytes) 2024-09-14T07:10:51.738 INFO:teuthology.orchestra.run.smithi145.stdout:5999 read 0x1d14800 thru 0x1d1aa83 (0x6284 bytes) 2024-09-14T07:10:51.739 INFO:teuthology.orchestra.run.smithi145.stdout:6000 punch from 0x16b200 to 0x16f400, (0x4200 bytes) 2024-09-14T07:10:51.769 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:51.766+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:51.793 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:51.793 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:51.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:51.870 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:51.866+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:51.937 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:51.937 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:51.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:51.971 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:51.970+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:51.989 INFO:teuthology.orchestra.run.smithi145.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-14T07:10:52.073 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:52.070+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:52.123 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:52.123 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:52.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:52.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:52.173 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:52.170+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:52.196 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.1. 2024-09-14T07:10:52.202 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:10:52.224 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-14T07:10:52.274 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:52.270+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:52.274 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:52.275 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:52.291 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:52.337 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:52.337 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:52.354 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:52.371 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:52.372 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:52.375 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:52.370+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:52.392 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:52.392 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:52.424 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:52.424 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:52.424 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:52.424 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:52.438 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:52.475 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:52.474+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:52.560 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:52.560 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:52.561 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:52.561 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:52.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:52.575 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:52.574+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:52.617 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:52.617 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:52.642 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:52.642 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:52.677 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:52.674+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 200.399609215, 2024-09-14T07:10:52.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:10:52.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:52.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 124.505616031, 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.710 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:10:52.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:10:52.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:52.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:10:52.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 124.50549615200001, 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.719 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:10:52.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:10:52.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 124.488271032, 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:10:52.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 101.815676645, 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:10:52.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:10:52.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2677 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1107968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041602551, 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10482514800000001, 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2677 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658511+0000", 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:10:52.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658514+0000", 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.43e-06 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658520+0000", 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8240000000000002e-06 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658525+0000", 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0309999999999998e-06 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763160+0000", 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10463465299999999 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763226+0000", 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6210999999999993e-05 2024-09-14T07:10:52.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763336+0000", 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010940799999999999 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2679 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1130496~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041580536, 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105172721, 2024-09-14T07:10:52.742 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2679 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658533+0000", 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6599999999999996e-07 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658535+0000", 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1160000000000002e-06 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658541+0000", 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658545+0000", 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2729999999999996e-06 2024-09-14T07:10:52.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763389+0000", 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10484404899999999 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763508+0000", 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118344 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763705+0000", 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019752699999999999 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2680 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1153024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041559601, 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105362471, 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2680 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658553+0000", 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658554+0000", 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.89e-07 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658556+0000", 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4080000000000001e-06 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658563+0000", 2024-09-14T07:10:52.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4450000000000001e-06 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658568+0000", 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763746+0000", 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105178023 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763803+0000", 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.728e-05 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763916+0000", 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011297 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:52.751 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.04153859100001, 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:52.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041504891, 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.756 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.760 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041484248, 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:10:52.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041239031, 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:10:52.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041207312, 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.770 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:10:52.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.04118555300001, 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.778 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.041153474, 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:52.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:10:52.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.039982268, 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:10:52.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:10:52.786 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.039959305, 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.787 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.039938015, 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.791 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:10:52.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:10:52.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.039917792, 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:10:52.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:10:52.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.912191688, 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:52.799 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:10:52.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.804 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:52.805 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:52.805 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:10:52.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:52.807 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:52.802+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:52.907 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:52.902+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:52.978 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:52.978 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:52.978 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:52.992 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:53.008 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:53.006+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:53.108 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:53.106+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:53.113 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:53.113 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:53.114 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:53.114 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:53.114 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:53.127 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:53.209 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:53.206+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:53.299 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.00017877600001, 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.300 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:10:53.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.14070318899999, 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:10:53.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.140600904, 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:10:53.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:10:53.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:53.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.315 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.441549063, 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:10:53.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.320 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.956537502, 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.324 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.95650343299999, 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:10:53.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:10:53.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:10:53.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.956460087, 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.329 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.956426564, 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:10:53.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:10:53.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.336 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.956379314, 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.337 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:10:53.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:10:53.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.956265066, 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.344 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.95615537099999, 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.348 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.956089228, 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:53.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:53.352 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.721013380000002, 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:53.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:10:53.357 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.434299944000003, 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.363 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.818284902000002, 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:10:53.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:10:53.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:10:53.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:53.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:53.369 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.818221203, 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:10:53.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:10:53.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:10:53.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.818177869000003, 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.375 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.380 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.818120989999997, 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:53.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:10:53.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.818054494999998, 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:10:53.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:10:53.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:10:53.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:10:53.392 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.815170158000001, 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:10:53.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.398 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:53.400 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:53.400 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:53.398+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:53.501 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:53.498+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:53.528 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:53.529 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:53.529 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:53.543 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:53.632 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:53.630+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:53.702 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:53.702+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:53.711 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:53.711 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:53.711 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:53.711 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:53.711 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:53.726 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:53.812 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:53.810+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:53.899 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 181.06449678000001, 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:53.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:10:53.904 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 181.06372243199999, 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:10:53.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:10:53.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 152.05253943400001, 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:10:53.909 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:10:53.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 151.75561122799999, 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:10:53.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:10:53.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:10:53.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.920 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.52070740400001, 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:10:53.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:10:53.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:10:53.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.520145613, 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:10:53.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:10:53.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:10:53.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.930 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.51965104600001, 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:10:53.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:10:53.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:10:53.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.519033627, 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:10:53.936 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:53.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:10:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:10:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.941 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.518605368, 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.942 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:10:53.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.946 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.518209504, 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.947 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:10:53.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.517617828, 2024-09-14T07:10:53.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:10:53.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.957 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.514043085, 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:10:53.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:10:53.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:10:53.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.51400373200001, 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:10:53.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.968 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.51396595200001, 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.969 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:10:53.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:10:53.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:10:53.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 107.513929249, 2024-09-14T07:10:53.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:10:53.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:10:53.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.924557315000001, 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:10:53.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.919884883000002, 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:10:53.985 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:53.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:10:53.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.919802390999997, 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.990 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.91964127, 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:53.995 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:10:53.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:10:53.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:10:53.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.919464361999999, 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:10:54.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:10:54.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:10:54.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:10:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:10:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:10:54.005 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:10:54.005 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:54.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:54.007 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:54.002+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:54.108 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:54.106+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:54.158 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:54.158 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:54.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:54.208 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:54.206+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:54.310 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:54.306+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:54.349 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:54.350 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:54.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:54.411 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:54.410+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:54.506 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.215353623, 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.185845992, 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.17719863299999, 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.51110629300001, 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.34437706700001, 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 148.41137605899999, 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.655932887000006, 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.655914964999994, 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.655902196, 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.655892019999996, 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.655881824999994, 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.655871805000004, 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 97.602370691000004, 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 66.442063179000002, 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 66.442031639999996, 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 66.442000328000006, 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 66.441960351000006, 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 66.432949531999995, 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 66.432339967999994, 2024-09-14T07:10:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 52.406598168999999, 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.593 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:54.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:54.595 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:54.590+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:54.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:54.735 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:54.734+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:54.797 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:54.794+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "MOSDPGPush(3.4 161/160 [PushOp(3:26c80f10:::rbd_data.11618d6e16a.0000000000000006:head, version: 104'450, data_included: [2381824~58880], data_size: 58880, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:26c80f10:::rbd_data.11618d6e16a.0000000000000006:head@104'450, size: 2621440, copy_subset: [(0, 2621440)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2621440, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:54.797345+0000", 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001039786, 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010920630000000001, 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797345+0000", 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797345+0000", 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797345+0000", 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7899999999999998e-07 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797380+0000", 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4235e-05 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797396+0000", 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6762999999999999e-05 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797413+0000", 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6704000000000001e-05 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797519+0000", 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106217 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:54.797533+0000", 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.415e-05 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:10:54.802 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:54.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:10:54.816 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:54.862 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:54.863 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:54.884 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:54.898 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:54.894+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:54.906 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:54.934 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:54.934 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:54.960 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:54.982 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:54.983 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:54.998 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:54.994+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:55.006 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:55.006 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:55.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:55.111 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:55.110+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:55.200 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:55.198+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:55.200 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:55.200 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:55.208 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:10:55.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:55.323 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:55.322+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:55.401 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:55.398+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:55.406 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:55.407 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:55.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:55.502 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:55.498+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:55.601 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:55.601 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:55.602 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:55.598+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:55.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:55.703 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:55.698+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:55.802 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:55.803 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:55.804 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:55.802+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:55.818 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:55.905 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:55.902+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:55.946 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:55.946 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:55.959 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:56.006 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:56.002+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:56.084 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:56.084 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:56.098 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:56.144 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:56.142+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:56.208 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:56.206+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:56.217 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:56.217 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:56.236 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:56.324 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:56.322+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:56.402 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:56.402 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:56.410 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:56.406+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:56.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:56.512 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:56.510+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:56.589 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:56.589 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:56.603 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:10:56.652 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:56.650+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:56.711 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:56.710+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:56.785 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:56.785 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:56.801 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:10:56.812 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:56.810+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:56.913 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:56.910+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:56.919 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:56.919 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:56.934 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:10:57.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:10:57.051 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:57.050+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:57.065 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:57.065 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:57.080 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:10:57.115 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:57.114+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:57.167 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:57.167 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:57.185 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:57.185 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:57.203 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:57.203 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:57.207 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:57.215 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:57.214+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:57.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:10:57.225 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:10:57.225 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:10:57.225 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-14T07:10:57.225 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:10:57.225 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:10:57.226 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-14T07:10:57.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:10:57.238 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:57.238 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:57.261 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:57.262 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:57.297 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:57.297 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:57.316 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:57.314+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:57.349 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:57.350 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "" 2024-09-14T07:10:57.350 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:57.358 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:57.358 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:57.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22321 3.10 3.b95aa970 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.090746670000000001, 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.090767168999999995, 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for peered", 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22321 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268340+0000", 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.784e-06 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268346+0000", 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268364+0000", 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7825999999999998e-05 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268373+0000", 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.577e-06 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268447+0000", 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3978000000000002e-05 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268463+0000", 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5478999999999999e-05 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:10:57.362 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:57.376 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:10:57.418 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:57.414+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:57.518 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:57.514+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:57.550 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:57.550 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:57.550 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:57.550 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:57.550 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:57.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:10:57.620 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:57.618+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:57.720 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:57.718+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.44813179499999, 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:10:57.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.41862416399999, 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:10:57.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.40997680500001, 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:10:57.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 155.74388446500001, 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:10:57.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:10:57.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:57.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 155.57715523900001, 2024-09-14T07:10:57.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:10:57.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.64415423099999, 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.763 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:10:57.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.888711059, 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.768 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:10:57.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:10:57.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.888693137, 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:57.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.888680368, 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.780 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.88867019200001, 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:57.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.888659997, 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:57.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:57.789 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.888649977, 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:10:57.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.835148863, 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:10:57.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.674841350999998, 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:10:57.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.674809812000007, 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.803 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:10:57.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:10:57.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:10:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.674778500000002, 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:57.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.674738523000002, 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:10:57.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:10:57.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3088 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:57.815 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.665727704000005, 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040560713999999998, 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3088 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071145+0000", 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071146+0000", 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071153+0000", 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-09-14T07:10:57.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071170+0000", 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7583000000000001e-05 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071183+0000", 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3104e-05 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111682+0000", 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04049883 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.111706+0000", 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3570999999999998e-05 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:10:57.819 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.665118140000004, 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:10:57.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:10:57.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.639376341000002, 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:10:57.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:57.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:57.827 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:57.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:10:57.829 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:10:57.830 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:57.826+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:57.929 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:57.926+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3.90012648 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.13802249799999999, 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.138067254, 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:57.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for peered", 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:57.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:10:58.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:10:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:10:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:10:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:10:58.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:10:58.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:10:58.003 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:58.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:10:58.056 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:58.054+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:58.130 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:58.126+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:58.179 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:58.180 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:58.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:10:58.236 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:58.234+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:58.333 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:58.330+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:58.365 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 156.683828667, 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:10:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9245 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.29285283, 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120282417, 2024-09-14T07:10:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9245 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070761+0000", 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070764+0000", 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-09-14T07:10:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070774+0000", 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2080000000000006e-06 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070780+0000", 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8129999999999998e-06 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191027+0000", 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12024617 2024-09-14T07:10:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191043+0000", 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6742999999999999e-05 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292823032, 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:10:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.29279075700001, 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.29273265400001, 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292691752, 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:10:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292629944, 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:10:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:10:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292595628, 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:10:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:10:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:10:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.29257695699999, 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:10:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:10:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292552921, 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:10:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:10:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:10:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.29253288699999, 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:10:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.29251805200001, 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292503426, 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292488925, 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:10:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.29247117, 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:10:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:10:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292456276, 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:10:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:10:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292442165, 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292420236, 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:10:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:10:58.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.292398925, 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:10:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 107.005048024, 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:10:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:10:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:10:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:58.448 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:58.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:10:58.450 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:58.446+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:58.551 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:58.546+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:58.623 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:58.623 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:58.623 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:10:58.623 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:58.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:10:58.652 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:58.650+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:58.753 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:58.750+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:58.820 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:10:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:10:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:10:58.820 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:58.835 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:10:58.853 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:58.850+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:58.954 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:58.950+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.70587041300001, 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.660880107, 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:10:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:10:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:10:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.35894765099999, 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:10:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:10:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.037619917, 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:10:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:10:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.600491518, 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:10:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.60048086, 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.60046584600001, 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:10:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.600454931, 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:10:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.60044342099999, 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:10:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:10:59.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.600430878, 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.59358428199999, 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:10:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:10:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.593536501, 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:10:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.593523034, 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:10:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.593512062, 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:10:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.5935013, 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:10:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:10:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.593485246, 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.593468675, 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:10:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.593456899, 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:10:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:10:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.593446222, 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:10:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:10:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:10:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.432269797000004, 2024-09-14T07:10:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:10:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:10:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:10:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:10:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:10:59.102 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:10:59.104 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:10:59.104 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:10:59.102+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:10:59.205 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:59.202+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:59.237 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:59.237 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:59.237 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:10:59.237 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:59.251 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:10:59.320 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:10:59.344 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:59.342+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:59.407 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:10:59.402+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:10:59.421 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:10:59.421 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:10:59.421 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:10:59.421 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:10:59.421 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:10:59.435 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:10:59.476 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:10:59.476 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:10:59.492 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:10:59.492 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:10:59.508 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:59.506+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:59.509 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:10:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:10:59.527 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:10:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:10:59.547 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:10:59.547 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:10:59.568 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:10:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:10:59.595 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:59.595 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:59.609 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:10:59.610 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:10:59.606+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:10:59.710 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:10:59.706+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:10:59.774 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:59.774 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:59.792 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:10:59.811 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:10:59.806+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:10:59.831 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:10:59.911 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:10:59.906+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:10:59.913 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:10:59.914 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:10:59.928 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:00.020 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:00.018+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:00.099 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:00.099 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:00.113 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:00.114 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:00.110+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:00.213 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:00.210+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:00.282 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:00.282 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:00.297 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:00.315 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:00.310+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:00.415 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:00.415 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:00.415 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:00.414+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:00.430 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:00.516 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:00.514+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:00.554 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:00.554 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:00.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:00.616 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:00.614+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:00.703 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:00.703 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:00.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:00.718 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:00.714+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:11:00.822 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:00.818+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:00.903 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:00.903 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:00.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:00.963 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:00.962+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:01.023 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:01.018+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:01.055 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:01.055 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:01.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:01.124 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:01.122+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:01.225 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:01.222+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:01.233 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:01.233 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:01.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:01.335 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:01.334+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:01.420 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:01.420 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:01.427 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:01.422+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:01.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:01.527 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:01.526+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:01.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:01.610 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:01.629 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:01.626+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:01.729 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:01.726+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:01.747 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:01.764 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:01.782 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:01.782 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:01.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:01.783 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:01.783 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:01.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:01.810 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:01.830 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:01.826+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:01.833 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:01.833 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:01.856 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:01.931 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:01.926+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:01.940 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:01.941 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:01.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:02.047 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:02.046+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:02.133 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:02.130+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 164.83216725099999, 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:02.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 164.787176945, 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:02.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:02.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:02.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.485244489, 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:02.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:02.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:02.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.163916755, 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:02.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:02.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.726788356, 2024-09-14T07:11:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:02.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:02.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.72677769800001, 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.166 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.72676268399999, 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:02.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.726751769, 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:02.171 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:02.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.726740259, 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:02.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.726727716, 2024-09-14T07:11:02.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.71988112, 2024-09-14T07:11:02.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.188 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.719833339, 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.719819872, 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:02.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:02.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.196 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.7198089, 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:02.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:02.200 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.719798138, 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:02.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.719782084, 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:02.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:02.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:02.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.719765513, 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:02.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:02.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.719753737, 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:02.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:02.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:02.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 115.71974306, 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:02.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:02.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.558566634999998, 2024-09-14T07:11:02.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:02.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:02.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.229 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:02.230 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:02.234 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:02.230+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:02.338 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:02.338+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:02.362 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:02.362 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:02.362 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:02.362 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:02.364 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:11:02.365 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:11:02.365 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T07:11:02.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:11:02.376 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:02.438 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:02.434+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:02.493 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:02.493 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:02.507 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:02.539 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:02.538+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 210.32893494199999, 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:02.631 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.637 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 134.43494175800001, 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:02.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:02.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:02.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 134.434821879, 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.646 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:02.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:02.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:02.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 134.41759675899999, 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.657 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 111.745002372, 2024-09-14T07:11:02.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:02.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:02.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:02.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:02.667 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.970864318, 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.668 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:02.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.97083061799999, 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:02.672 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.676 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.97080997499999, 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:02.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.680 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.97056475799999, 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.681 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:02.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.970533039, 2024-09-14T07:11:02.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:02.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.689 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.97051128, 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.690 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:02.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.970479201, 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:02.694 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:02.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:02.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.96930799499999, 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.969285032, 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:02.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.969263742, 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.707 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:02.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:02.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.969243519, 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.712 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.841517414999998, 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:02.716 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:02.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:02.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:02.721 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.8422723019999996, 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:02.722 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:02.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.8422249100000005, 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.8421833830000001, 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:02.732 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:02.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:02.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:02.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:02.737 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 169, 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:11:01.779073+0000", 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:11:02.739 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:10:52.180916+0000", 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5806080, 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "noscrub", 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:11:02.740 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "luminous", 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:11:02.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:11:02.742 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:11:02.743 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:11:02.744 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:11:02.745 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 5, 2024-09-14T07:11:02.746 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 5, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.40000000596046448, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.75441241264343262, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.75 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:11:02.747 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:11:02.748 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:11:02.749 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:11:02.750 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:11:02.751 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.8700000047683716, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.8700000047683716, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1.2000000476837158, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.75441241264343262, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-14T07:11:02.752 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.75 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:11:02.753 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:11:02.754 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "169", 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 46, 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 169, 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:11:02.755 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:11:02.756 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:11:02.757 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.4099999666213989, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.4099999666213989, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1.2000000476837158, 2024-09-14T07:11:02.758 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.690000057220459, 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.690000057220459, 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.75441241264343262, 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.75 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:02.759 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.5264739990234375, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 164, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:02.764 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 166, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:02.765 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.766 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.767 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.768 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.769 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:02.770 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 165, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 163, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:02.776 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.780 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:02.781 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 163, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.782 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.783 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.786 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:02.787 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.5415802001953125, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 159, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:11:02.788 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:11:02.789 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:11:02.790 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:11:02.791 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:11:02.792 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:02.793 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.794 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:02.795 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:11:02.796 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [ 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: "pgid": "3.15", 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: 3, 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: 2 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:11:02.798 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [ 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "snaps": [ 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.799 INFO:teuthology.orchestra.run.smithi005.stdout: "begin": 44, 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: "length": 3 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [ 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: "snaps": [ 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: "begin": 44, 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: "length": 1 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.800 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:02.801 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:02.802 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:02.802 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:02.738+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:02.803 INFO:tasks.thrashosds.thrasher:j is {'epoch': 169, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:11:01.779073+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:10:52.180916+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7544124126434326, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.75}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8700000047683716, 'score_stable': 1.8700000047683716, 'optimal_score': 1.2000000476837158, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.7544124126434326, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.75}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 169, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.409999966621399, 'score_stable': 1.409999966621399, 'optimal_score': 1.2000000476837158, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.7544124126434326, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.75}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5264739990234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 165, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 163, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 163, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 0.5415802001953125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 159, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.15', 'osds': [3, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 44, 'length': 3}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 44, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:11:02.803 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.15 2024-09-14T07:11:02.804 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.15 2024-09-14T07:11:02.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:11:02.848 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:02.846+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:02.938 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:02.938 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:02.938 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:02.940 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:02.938+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:02.955 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:03.044 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:03.042+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:03.124 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:03.125 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:03.125 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:03.125 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:03.125 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:03.139 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:03.143 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:03.142+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:03.245 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:03.242+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.005365055, 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.305 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:03.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.14588946800001, 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.311 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:03.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.14578718300001, 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.316 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:03.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:03.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 152.44673534200001, 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.321 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.325 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.961723781, 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:03.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.96168971199999, 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.330 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:03.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.961646366, 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.334 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:03.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.961612843, 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:03.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:03.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.961565593, 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.961451345, 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:03.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.96134164999999, 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:03.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.353 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 116.961275507, 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.354 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:03.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:03.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 60.726199659000002, 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:03.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:03.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.439486223000003, 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:03.363 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:03.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:03.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:03.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:03.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.823471181000002, 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:03.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:03.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:03.375 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.823407482, 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:03.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:03.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:03.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:03.381 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.823364148000003, 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:03.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:03.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:03.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:03.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.823307268999997, 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:03.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:03.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.823240773999999, 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.393 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:03.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:03.398 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.820356437000001, 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.404 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:03.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:03.406 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:03.402+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:03.506 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:03.502+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:03.574 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:03.574 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:03.574 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:03.574 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:03.589 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:03.636 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:03.634+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:03.707 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:03.706+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:03.713 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:03.714 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:03.714 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:03.714 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:03.714 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:03.728 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:clear 3.15 pg_upmap mapping 2024-09-14T07:11:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:11:03.819 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:03.818+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:03.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 191.07180658600001, 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 191.07103223799999, 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.05984924000001, 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 161.76292103399999, 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.52801721, 2024-09-14T07:11:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.52745541900001, 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.526960852, 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.52634343299999, 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.525915174, 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:03.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:03.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.52551930999999, 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:03.948 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.952 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.52492763399999, 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.52135289100001, 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:03.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:03.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:03.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.521313538, 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:03.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.521275758, 2024-09-14T07:11:03.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:03.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 117.521239055, 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:03.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:03.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.931867121000003, 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:03.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:03.978 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.927194688999997, 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:03.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.927112197, 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:03.987 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.926951076000002, 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:03.991 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.926774168000001, 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:03.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:03.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:03.996 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:03.998 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:03.994+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:03.998 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:04.027 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:04.027 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:04.044 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:04.061 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:04.061 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:04.082 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:04.082 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:04.098 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:04.094+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:04.103 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:04.103 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:04.116 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:04.116 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:04.125 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:04.125 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:04.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:04.199 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:04.198+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:04.300 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:04.298+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:04.313 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:04.313 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:04.327 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:04.402 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:04.398+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:04.455 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:04.455 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:04.469 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:04.515 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:04.514+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:04.589 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:04.589 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:04.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:04.604 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:04.602+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:04.705 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:04.702+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:04.775 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:04.776 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:04.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:04.835 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:04.834+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:04.907 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:04.906+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:04.920 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:04.920 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:04.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:05.023 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:05.022+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:05.101 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:05.101 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:05.108 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:05.106+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:05.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:05.209 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:05.206+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:05.287 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:05.288 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:05.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:05.310 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:05.306+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:05.411 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:05.410+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:05.475 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:05.476 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:05.490 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:05.536 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:05.534+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:05.613 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:05.610+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:05.623 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:05.623 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:05.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:05.714 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:05.714+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:05.805 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:05.805 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:05.815 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:05.810+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:05.820 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:05.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:11:05.915 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:05.914+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:05.987 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:05.987 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:06.001 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:06.048 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:06.046+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:06.117 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:06.114+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:06.125 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:06.125 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:06.139 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:06.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:06.218 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:06.214+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:06.306 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:06.319 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:06.318+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:06.325 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:06.325 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:06.331 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:06.331 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:06.343 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:06.349 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:06.364 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:06.388 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:06.406 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:06.407 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:06.420 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:06.418+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 169.181671231, 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.322195644, 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.32209335900001, 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.62304151800001, 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.138029957, 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.13799588800001, 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.13795254199999, 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.13791901899999, 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.137871769, 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.137757521, 2024-09-14T07:11:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.13764782600001, 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.13758168299999, 2024-09-14T07:11:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.902505834999999, 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 54.615792399, 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.999777356999999, 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.556 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:06.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:06.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.999713657999997, 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:06.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:06.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:06.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:06.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.999670324, 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:06.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:06.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:06.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.999613445000001, 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:06.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:06.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:06.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.999546950000003, 2024-09-14T07:11:06.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.996662612999998, 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:06.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:06.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:06.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:06.599 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:06.601 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:06.601 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:06.598+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:06.702 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:06.698+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:06.771 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:06.771 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:06.771 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:06.771 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:06.786 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:06.828 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:06.826+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:11:06.904 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:06.902+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:06.956 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:06.956 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:06.956 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:06.956 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:06.956 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:06.972 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:07.019 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:07.018+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:07.105 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:07.102+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 194.27685343499999, 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:07.110 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:07.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:07.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:07.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:07.116 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 194.276079087, 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.117 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:07.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.122 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.26489608899999, 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:07.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:07.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:07.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 164.967967883, 2024-09-14T07:11:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:07.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:07.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:07.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:07.136 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.733064059, 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:07.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.141 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.732502268, 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:07.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.732007701, 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.147 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:07.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.73139028200001, 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.152 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:07.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.730962023, 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:07.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:07.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:07.161 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.73056615900001, 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:07.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:07.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:07.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.72997448300001, 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:07.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.72639974000001, 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:07.172 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:07.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:07.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.726360387, 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.178 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:07.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:07.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.726322607, 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.183 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:07.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.187 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 120.72628590399999, 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.188 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:07.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:07.192 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.136913970000002, 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.193 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:07.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.132241538000002, 2024-09-14T07:11:07.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:07.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.132159045999998, 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:07.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:07.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.131997925, 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:07.209 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:07.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:07.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:07.214 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.131821017, 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.215 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:07.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:07.220 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:07.220 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:07.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:07.222 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:07.218+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:07.323 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:07.318+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:07.356 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:07.356 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:07.356 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:07.370 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:07.424 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:07.422+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:07.525 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:07.522+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:07.536 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:07.537 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:07.537 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:07.537 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:07.537 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:07.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:07.626 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:07.622+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:07.678 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 170.38875996600001, 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:11:07.679 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:11:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:11:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 170.35925233500001, 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:11:07.686 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:11:07.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 170.350604976, 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:11:07.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:11:07.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 165.68451263599999, 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:11:07.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 165.51778340999999, 2024-09-14T07:11:07.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:11:07.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:11:07.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:11:07.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:11:07.707 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.584782402, 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:11:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:07.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:11:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:07.713 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.82933923, 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:11:07.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.829321308, 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:11:07.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.829308539, 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:11:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:11:07.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.728 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.82929836300001, 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:11:07.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:11:07.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.82928816800001, 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.734 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:11:07.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:11:07.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.829278148, 2024-09-14T07:11:07.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:11:07.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:07.744 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 110.775777034, 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:07.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:11:07.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:07.749 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.615469521999998, 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.750 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:11:07.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:11:07.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.615437983000007, 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:11:07.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:11:07.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.615406671000002, 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:07.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:11:07.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:11:07.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.615366694000002, 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:11:07.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 79.605746311000004, 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:11:07.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.580004512000002, 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.779 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22321 3.10 3:0e955a9d:::rbd_data.1161b7591419.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.40916313, 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.52034672699999995, 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22321 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:07.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268340+0000", 2024-09-14T07:11:07.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.784e-06 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268346+0000", 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268364+0000", 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7825999999999998e-05 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268373+0000", 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.577e-06 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268447+0000", 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3978000000000002e-05 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268463+0000", 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5478999999999999e-05 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.781808+0000", 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.51334471599999998 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.782098+0000", 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00029031499999999997 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.782282+0000", 2024-09-14T07:11:07.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018428300000000001 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788372+0000", 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0060903279999999999 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788543+0000", 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017061499999999999 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788622+0000", 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9059000000000003e-05 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788685+0000", 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2395000000000001e-05 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:07.787 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:07.787 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:07.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:07.789 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:07.786+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:07.889 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:07.886+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:07.920 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:07.921 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:07.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:07.990 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:07.986+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:08.068 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:08.068 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:08.068 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:08.068 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:08.069 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:08.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:08.131 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:08.130+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:08.192 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:08.190+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:08.263 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:08.263 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:08.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:08.263 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 166.581385786, 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:08.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:08.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190380151, 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190347876, 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.272 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:08.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.275 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190289773, 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:08.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.19024887099999, 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:08.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:08.283 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190187063, 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190152747, 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:08.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.19013407600001, 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:08.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.19011003999999, 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:08.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:08.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:08.298 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.19009000600001, 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:08.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190075171, 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:08.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.19006054499999, 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:08.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190046044, 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.190028289, 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:08.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.19001339499999, 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:08.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:08.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.189999284, 2024-09-14T07:11:08.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:08.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:08.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.189977355, 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 121.189956044, 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 116.902605143, 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.400936754, 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:08.348 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:08.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:08.350 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:08.346+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:08.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:08.451 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:08.450+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:08.482 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:08.482 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:08.482 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:08.482 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:08.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:08.552 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:08.550+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:08.569 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:08.586 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:08.588 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:08.603 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:08.618 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:08.618 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:08.624 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:08.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:08.646 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:08.646 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:08.652 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:08.650+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:08.667 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:08.751 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:08.752 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:08.753 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:08.750+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:08.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:08.839 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:11:08.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:11:08.839 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T07:11:08.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:11:08.853 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:08.850+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:08.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:11:08.903 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:08.903 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:08.920 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:08.964 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:08.962+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:09.055 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:09.054+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:09.097 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:09.097 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:09.112 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:09.156 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:09.154+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 176, 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "717e3012-acdf-42c1-a4e0-d4c94afd9f15", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-09-14T07:07:18.019909+0000", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-09-14T07:11:08.839353+0000", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-09-14T07:07:31.282745+0000", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-09-14T07:10:52.180916+0000", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5799936, 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-09-14T07:11:09.193 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 6, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-09-14T07:11:09.194 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:31.246122+0000", 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:11:09.195 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "15", 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:11:09.196 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:11:09.197 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:11:09.198 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:11:09.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:11:09.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-09-14T07:11:09.200 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 5, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 5, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.40000000596046448, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.75441241264343262, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.75 2024-09-14T07:11:09.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:34.989638+0000", 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:11:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.203 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "19", 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 2, 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 19, 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:11:09.204 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:11:09.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.8700000047683716, 2024-09-14T07:11:09.206 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.8700000047683716, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1.2000000476837158, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.75441241264343262, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.75 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-09-14T07:07:45.828508+0000", 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-09-14T07:11:09.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "is_stretch_pool": false, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-09-14T07:11:09.208 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "176", 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:11:09.209 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 46, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 176, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-09-14T07:11:09.210 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:11:09.211 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.4099999666213989, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.4099999666213989, 2024-09-14T07:11:09.212 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1.2000000476837158, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.690000057220459, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.690000057220459, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.75441241264343262, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.75 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "9c97eae7-11c5-4962-a62d-04d698e50114", 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:09.213 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.5264739990234375, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 164, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-09-14T07:11:09.215 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1673964337 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.217 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/1673964337", 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/1673964337", 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/1673964337", 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/1673964337", 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "91d33ac7-8429-4237-bb93-1259b318642a", 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:09.218 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 166, 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.219 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-09-14T07:11:09.220 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1704459401 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/1704459401", 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/1704459401", 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6807/1704459401", 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/1704459401", 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "02202e5a-e516-4438-9378-7fad80297e64", 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:09.223 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 165, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.225 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2565128645 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2565128645", 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2565128645", 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2565128645", 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2565128645", 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57a45940-90af-484d-9570-8de1c6b2252b", 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:09.228 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 174, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6800", 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6801", 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6802", 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6803", 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6806", 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6807", 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6804", 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.232 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6805", 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2226792373 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6801/2226792373", 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6803/2226792373", 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6807/2226792373", 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6805/2226792373", 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:11:09.233 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "8150dd96-bfa0-404f-ae37-f18897bcc93e", 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 163, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.234 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6816", 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6817", 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6818", 2024-09-14T07:11:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6819", 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6822", 2024-09-14T07:11:09.236 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6823", 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6820", 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6821", 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 940255113 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6817/940255113", 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6819/940255113", 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6823/940255113", 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6821/940255113", 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "59d63d5d-8ef9-43b2-8d7f-49e8e8385435", 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.5415802001953125, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 12, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 159, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6808", 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6809", 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6810", 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6811", 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.241 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6814", 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6815", 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6812", 2024-09-14T07:11:09.242 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.110:6813", 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4012173203 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.110:6809/4012173203", 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.110:6811/4012173203", 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.110:6815/4012173203", 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.110:6813/4012173203", 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-09-14T07:11:09.243 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.952391+0000", 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.244 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.101881+0000", 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:09.245 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.869317+0000", 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.961339+0000", 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:09.246 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:26.296733+0000", 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540719139924082687, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-09-14T07:07:25.723820+0000", 2024-09-14T07:11:09.247 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-09-14T07:11:09.248 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [ 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: "snaps": [ 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: "begin": 39, 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: "length": 7 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [ 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: "snaps": [ 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: "begin": 39, 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: "length": 1 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-09-14T07:11:09.250 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:09.251 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:09.252 INFO:tasks.thrashosds.thrasher:j is {'epoch': 176, 'fsid': '717e3012-acdf-42c1-a4e0-d4c94afd9f15', 'created': '2024-09-14T07:07:18.019909+0000', 'modified': '2024-09-14T07:11:08.839353+0000', 'last_up_change': '2024-09-14T07:07:31.282745+0000', 'last_in_change': '2024-09-14T07:10:52.180916+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T07:07:31.246122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.4000000059604645, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7544124126434326, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.75}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T07:07:34.989638+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8700000047683716, 'score_stable': 1.8700000047683716, 'optimal_score': 1.2000000476837158, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.7544124126434326, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.75}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T07:07:45.828508+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 176, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.409999966621399, 'score_stable': 1.409999966621399, 'optimal_score': 1.2000000476837158, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 0.7544124126434326, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.75}}], 'osds': [{'osd': 0, 'uuid': '9c97eae7-11c5-4962-a62d-04d698e50114', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5264739990234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 164, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 1673964337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 1673964337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 1673964337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 1673964337}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 1673964337}]}, 'public_addr': '172.21.15.5:6817/1673964337', 'cluster_addr': '172.21.15.5:6819/1673964337', 'heartbeat_back_addr': '172.21.15.5:6823/1673964337', 'heartbeat_front_addr': '172.21.15.5:6821/1673964337', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '91d33ac7-8429-4237-bb93-1259b318642a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 1704459401}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 1704459401}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6807', 'nonce': 1704459401}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 1704459401}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 1704459401}]}, 'public_addr': '172.21.15.5:6801/1704459401', 'cluster_addr': '172.21.15.5:6803/1704459401', 'heartbeat_back_addr': '172.21.15.5:6807/1704459401', 'heartbeat_front_addr': '172.21.15.5:6805/1704459401', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '02202e5a-e516-4438-9378-7fad80297e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 165, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6808', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2565128645}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2565128645}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2565128645}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2565128645}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2565128645}]}, 'public_addr': '172.21.15.5:6809/2565128645', 'cluster_addr': '172.21.15.5:6811/2565128645', 'heartbeat_back_addr': '172.21.15.5:6815/2565128645', 'heartbeat_front_addr': '172.21.15.5:6813/2565128645', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '57a45940-90af-484d-9570-8de1c6b2252b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 174, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 2226792373}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 2226792373}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 2226792373}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 2226792373}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 2226792373}]}, 'public_addr': '172.21.15.110:6801/2226792373', 'cluster_addr': '172.21.15.110:6803/2226792373', 'heartbeat_back_addr': '172.21.15.110:6807/2226792373', 'heartbeat_front_addr': '172.21.15.110:6805/2226792373', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '8150dd96-bfa0-404f-ae37-f18897bcc93e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 163, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6816', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 940255113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 940255113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 940255113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 940255113}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 940255113}]}, 'public_addr': '172.21.15.110:6817/940255113', 'cluster_addr': '172.21.15.110:6819/940255113', 'heartbeat_back_addr': '172.21.15.110:6823/940255113', 'heartbeat_front_addr': '172.21.15.110:6821/940255113', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '59d63d5d-8ef9-43b2-8d7f-49e8e8385435', 'up': 1, 'in': 1, 'weight': 0.5415802001953125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 159, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 4012173203}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 4012173203}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6815', 'nonce': 4012173203}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 4012173203}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 4012173203}]}, 'public_addr': '172.21.15.110:6809/4012173203', 'cluster_addr': '172.21.15.110:6811/4012173203', 'heartbeat_back_addr': '172.21.15.110:6815/4012173203', 'heartbeat_front_addr': '172.21.15.110:6813/4012173203', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.952391+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.101881+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.869317+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.961339+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:26.296733+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T07:07:25.723820+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 39, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 39, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:11:09.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:11:09.257 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:09.254+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:09.276 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:09.276 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:09.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:09.376 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:09.374+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:09.456 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:09.456 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:09.458 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:09.454+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:09.470 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:11:09.561 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:09.554+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:09.578 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-14T07:11:09.258774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1784612013,"num_objects":1486,"num_object_clones":263,"num_object_copies":2972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1486,"num_whiteouts":460,"num_read":332175,"num_read_kb":1108387,"num_write":20072,"num_write_kb":331040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3279,"num_bytes_recovered":254172112,"num_keys_recovered":643,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16780,"ondisk_log_size":16780,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":399112,"kb_used_data":56496,"kb_used_omap":9,"kb_used_meta":342454,"kb_avail":565831928,"statfs":{"total":579820584960,"available":579411894272,"internally_reserved":0,"allocated":57851904,"data_stored":398148914,"data_compressed":18912758,"data_compressed_allocated":36556800,"data_compressed_original":387628032,"omap_allocated":9568,"internal_metadata":350673568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-127634271,"num_objects":-111,"num_object_clones":-18,"num_object_copies":-222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-111,"num_whiteouts":-13,"num_read":896,"num_read_kb":764,"num_write":191,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001040"},"pg_stats":[{"pgid":"3.1f","version":"170'354","reported_seq":6720,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.808557+0000","last_change":"2024-09-14T07:11:03.817943+0000","last_active":"2024-09-14T07:11:04.808557+0000","last_peered":"2024-09-14T07:11:04.808557+0000","last_clean":"2024-09-14T07:11:04.808557+0000","last_became_active":"2024-09-14T07:09:14.777937+0000","last_became_peered":"2024-09-14T07:09:14.777937+0000","last_unstale":"2024-09-14T07:11:04.808557+0000","last_undegraded":"2024-09-14T07:11:04.808557+0000","last_fullsized":"2024-09-14T07:11:04.808557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:01:49.970256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033748850000000002,"stat_sum":{"num_bytes":73957392,"num_objects":39,"num_object_clones":4,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":6401,"num_read_kb":50527,"num_write":536,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.1e","version":"170'643","reported_seq":12622,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.811244+0000","last_change":"2024-09-14T07:11:03.813050+0000","last_active":"2024-09-14T07:11:04.811244+0000","last_peered":"2024-09-14T07:11:04.811244+0000","last_clean":"2024-09-14T07:11:04.811244+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:11:04.811244+0000","last_undegraded":"2024-09-14T07:11:04.811244+0000","last_fullsized":"2024-09-14T07:11:04.811244+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:40:37.077949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0050346519999999997,"stat_sum":{"num_bytes":72017798,"num_objects":65,"num_object_clones":13,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":19,"num_read":14097,"num_read_kb":59638,"num_write":888,"num_write_kb":21279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.1d","version":"170'1437","reported_seq":15806,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.831432+0000","last_change":"2024-09-14T07:11:03.812809+0000","last_active":"2024-09-14T07:11:04.831432+0000","last_peered":"2024-09-14T07:11:04.831432+0000","last_clean":"2024-09-14T07:11:04.831432+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:11:04.831432+0000","last_undegraded":"2024-09-14T07:11:04.831432+0000","last_fullsized":"2024-09-14T07:11:04.831432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1437,"log_dups_size":0,"ondisk_log_size":1437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:06.633522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018748650000000001,"stat_sum":{"num_bytes":15393015,"num_objects":52,"num_object_clones":5,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":15,"num_read":19913,"num_read_kb":35468,"num_write":1554,"num_write_kb":5634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.1c","version":"170'390","reported_seq":20215,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.831055+0000","last_change":"2024-09-14T07:11:03.812379+0000","last_active":"2024-09-14T07:11:04.831055+0000","last_peered":"2024-09-14T07:11:04.831055+0000","last_clean":"2024-09-14T07:11:04.831055+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:11:04.831055+0000","last_undegraded":"2024-09-14T07:11:04.831055+0000","last_fullsized":"2024-09-14T07:11:04.831055+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:09:00.726247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014744199999999999,"stat_sum":{"num_bytes":43666484,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":10,"num_read":20629,"num_read_kb":46260,"num_write":374,"num_write_kb":7490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.1b","version":"170'523","reported_seq":8418,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.808379+0000","last_change":"2024-09-14T07:11:03.817172+0000","last_active":"2024-09-14T07:11:04.808379+0000","last_peered":"2024-09-14T07:11:04.808379+0000","last_clean":"2024-09-14T07:11:04.808379+0000","last_became_active":"2024-09-14T07:09:14.776333+0000","last_became_peered":"2024-09-14T07:09:14.776333+0000","last_unstale":"2024-09-14T07:11:04.808379+0000","last_undegraded":"2024-09-14T07:11:04.808379+0000","last_fullsized":"2024-09-14T07:11:04.808379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:38:58.298731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031779680000000002,"stat_sum":{"num_bytes":103910254,"num_objects":51,"num_object_clones":9,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":18,"num_read":8739,"num_read_kb":22821,"num_write":681,"num_write_kb":11075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.1a","version":"171'353","reported_seq":29535,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T07:11:07.053094+0000","last_change":"2024-09-14T07:11:06.838608+0000","last_active":"2024-09-14T07:11:07.053094+0000","last_peered":"2024-09-14T07:11:07.053094+0000","last_clean":"2024-09-14T07:11:07.053094+0000","last_became_active":"2024-09-14T07:10:56.234972+0000","last_became_peered":"2024-09-14T07:10:56.234972+0000","last_unstale":"2024-09-14T07:11:07.053094+0000","last_undegraded":"2024-09-14T07:11:07.053094+0000","last_fullsized":"2024-09-14T07:11:07.053094+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:22:56.804834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001185721,"stat_sum":{"num_bytes":84041389,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":12,"num_read":29428,"num_read_kb":50314,"num_write":395,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":37625216,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"29","length":"6"}]},{"pgid":"3.19","version":"170'1279","reported_seq":10257,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.828912+0000","last_change":"2024-09-14T07:11:03.941528+0000","last_active":"2024-09-14T07:11:04.828912+0000","last_peered":"2024-09-14T07:11:04.828912+0000","last_clean":"2024-09-14T07:11:04.828912+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:11:04.828912+0000","last_undegraded":"2024-09-14T07:11:04.828912+0000","last_fullsized":"2024-09-14T07:11:04.828912+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1279,"log_dups_size":0,"ondisk_log_size":1279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:40:40.769789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020712339999999999,"stat_sum":{"num_bytes":35154212,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":15,"num_read":14516,"num_read_kb":35904,"num_write":2080,"num_write_kb":9602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.18","version":"170'369","reported_seq":5074,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.808190+0000","last_change":"2024-09-14T07:11:03.818464+0000","last_active":"2024-09-14T07:11:04.808190+0000","last_peered":"2024-09-14T07:11:04.808190+0000","last_clean":"2024-09-14T07:11:04.808190+0000","last_became_active":"2024-09-14T07:10:41.119373+0000","last_became_peered":"2024-09-14T07:10:41.119373+0000","last_unstale":"2024-09-14T07:11:04.808190+0000","last_undegraded":"2024-09-14T07:11:04.808190+0000","last_fullsized":"2024-09-14T07:11:04.808190+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:55:51.577472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039262350000000001,"stat_sum":{"num_bytes":81316040,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":4676,"num_read_kb":15102,"num_write":448,"num_write_kb":15526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.17","version":"167'214","reported_seq":3514,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.831349+0000","last_change":"2024-09-14T07:11:03.812069+0000","last_active":"2024-09-14T07:11:04.831349+0000","last_peered":"2024-09-14T07:11:04.831349+0000","last_clean":"2024-09-14T07:11:04.831349+0000","last_became_active":"2024-09-14T07:09:11.348905+0000","last_became_peered":"2024-09-14T07:09:11.348905+0000","last_unstale":"2024-09-14T07:11:04.831349+0000","last_undegraded":"2024-09-14T07:11:04.831349+0000","last_fullsized":"2024-09-14T07:11:04.831349+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:49:19.250438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011860760000000001,"stat_sum":{"num_bytes":41792550,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":2825,"num_read_kb":20413,"num_write":224,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.16","version":"170'550","reported_seq":9583,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810682+0000","last_change":"2024-09-14T07:11:03.812456+0000","last_active":"2024-09-14T07:11:04.810682+0000","last_peered":"2024-09-14T07:11:04.810682+0000","last_clean":"2024-09-14T07:11:04.810682+0000","last_became_active":"2024-09-14T07:10:57.222981+0000","last_became_peered":"2024-09-14T07:10:57.222981+0000","last_unstale":"2024-09-14T07:11:04.810682+0000","last_undegraded":"2024-09-14T07:11:04.810682+0000","last_fullsized":"2024-09-14T07:11:04.810682+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:51.403962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048421569999999997,"stat_sum":{"num_bytes":51489996,"num_objects":49,"num_object_clones":5,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":13,"num_read":10044,"num_read_kb":43207,"num_write":661,"num_write_kb":6102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":205,"num_bytes_recovered":4158872,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.15","version":"170'432","reported_seq":6209,"reported_epoch":171,"state":"remapped+peering","last_fresh":"2024-09-14T07:11:03.807850+0000","last_change":"2024-09-14T07:11:04.811841+0000","last_active":"2024-09-14T07:11:03.807850+0000","last_peered":"2024-09-14T07:11:03.080765+0000","last_clean":"2024-09-14T07:11:03.080765+0000","last_became_active":"2024-09-14T07:10:33.076531+0000","last_became_peered":"2024-09-14T07:10:33.076531+0000","last_unstale":"2024-09-14T07:11:03.807850+0000","last_undegraded":"2024-09-14T07:11:03.807850+0000","last_fullsized":"2024-09-14T07:11:03.807850+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020997210000000001,"stat_sum":{"num_bytes":71055409,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":14,"num_read":7023,"num_read_kb":46887,"num_write":478,"num_write_kb":16655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":28834451,"num_keys_recovered":47,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2b","length":"4"}]},{"pgid":"3.14","version":"170'644","reported_seq":10538,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810525+0000","last_change":"2024-09-14T07:11:03.810334+0000","last_active":"2024-09-14T07:11:04.810525+0000","last_peered":"2024-09-14T07:11:04.810525+0000","last_clean":"2024-09-14T07:11:04.810525+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:11:04.810525+0000","last_undegraded":"2024-09-14T07:11:04.810525+0000","last_fullsized":"2024-09-14T07:11:04.810525+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:43:42.972686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031632460000000002,"stat_sum":{"num_bytes":64966259,"num_objects":57,"num_object_clones":10,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":20,"num_read":11784,"num_read_kb":38791,"num_write":841,"num_write_kb":14334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":339,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.811510+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:11:04.811510+0000","last_peered":"2024-09-14T07:11:04.811510+0000","last_clean":"2024-09-14T07:11:04.811510+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:11:04.811510+0000","last_undegraded":"2024-09-14T07:11:04.811510+0000","last_fullsized":"2024-09-14T07:11:04.811510+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"170'548","reported_seq":12175,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.808541+0000","last_change":"2024-09-14T07:11:03.817643+0000","last_active":"2024-09-14T07:11:04.808541+0000","last_peered":"2024-09-14T07:11:04.808541+0000","last_clean":"2024-09-14T07:11:04.808541+0000","last_became_active":"2024-09-14T07:10:56.234947+0000","last_became_peered":"2024-09-14T07:10:56.234947+0000","last_unstale":"2024-09-14T07:11:04.808541+0000","last_undegraded":"2024-09-14T07:11:04.808541+0000","last_fullsized":"2024-09-14T07:11:04.808541+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:00:58.508316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034668440000000002,"stat_sum":{"num_bytes":29248512,"num_objects":43,"num_object_clones":4,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":8,"num_read":13575,"num_read_kb":34772,"num_write":642,"num_write_kb":5091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.829181+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:11:04.829181+0000","last_peered":"2024-09-14T07:11:04.829181+0000","last_clean":"2024-09-14T07:11:04.829181+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:11:04.829181+0000","last_undegraded":"2024-09-14T07:11:04.829181+0000","last_fullsized":"2024-09-14T07:11:04.829181+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"171'579","reported_seq":7900,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.829181+0000","last_change":"2024-09-14T07:11:03.950870+0000","last_active":"2024-09-14T07:11:04.829181+0000","last_peered":"2024-09-14T07:11:04.829181+0000","last_clean":"2024-09-14T07:11:04.829181+0000","last_became_active":"2024-09-14T07:10:57.225048+0000","last_became_peered":"2024-09-14T07:10:57.225048+0000","last_unstale":"2024-09-14T07:11:04.829181+0000","last_undegraded":"2024-09-14T07:11:04.829181+0000","last_fullsized":"2024-09-14T07:11:04.829181+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:25:47.605826+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010174404,"stat_sum":{"num_bytes":40555606,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":9453,"num_read_kb":40838,"num_write":644,"num_write_kb":6313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":339,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810860+0000","last_change":"2024-09-14T07:09:11.347249+0000","last_active":"2024-09-14T07:11:04.810860+0000","last_peered":"2024-09-14T07:11:04.810860+0000","last_clean":"2024-09-14T07:11:04.810860+0000","last_became_active":"2024-09-14T07:09:11.346502+0000","last_became_peered":"2024-09-14T07:09:11.346502+0000","last_unstale":"2024-09-14T07:11:04.810860+0000","last_undegraded":"2024-09-14T07:11:04.810860+0000","last_fullsized":"2024-09-14T07:11:04.810860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:27.767446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"170'621","reported_seq":6252,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.829059+0000","last_change":"2024-09-14T07:11:03.942874+0000","last_active":"2024-09-14T07:11:04.829059+0000","last_peered":"2024-09-14T07:11:04.829059+0000","last_clean":"2024-09-14T07:11:04.829059+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:11:04.829059+0000","last_undegraded":"2024-09-14T07:11:04.829059+0000","last_fullsized":"2024-09-14T07:11:04.829059+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:57:29.493423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031016849999999999,"stat_sum":{"num_bytes":12931143,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":7504,"num_read_kb":17122,"num_write":815,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.829177+0000","last_change":"2024-09-14T07:10:52.790756+0000","last_active":"2024-09-14T07:11:04.829177+0000","last_peered":"2024-09-14T07:11:04.829177+0000","last_clean":"2024-09-14T07:11:04.829177+0000","last_became_active":"2024-09-14T07:10:52.790271+0000","last_became_peered":"2024-09-14T07:10:52.790271+0000","last_unstale":"2024-09-14T07:11:04.829177+0000","last_undegraded":"2024-09-14T07:11:04.829177+0000","last_fullsized":"2024-09-14T07:11:04.829177+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:38:49.537802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"171'684","reported_seq":3681,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.831170+0000","last_change":"2024-09-14T07:11:03.818339+0000","last_active":"2024-09-14T07:11:04.831170+0000","last_peered":"2024-09-14T07:11:04.831170+0000","last_clean":"2024-09-14T07:11:04.831170+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:11:04.831170+0000","last_undegraded":"2024-09-14T07:11:04.831170+0000","last_fullsized":"2024-09-14T07:11:04.831170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:49:40.049044+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0071283199999999996,"stat_sum":{"num_bytes":16480557,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":8,"num_read":5127,"num_read_kb":19644,"num_write":1050,"num_write_kb":5788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.3","version":"174'305","reported_seq":9372,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T07:11:07.010364+0000","last_change":"2024-09-14T07:11:06.843031+0000","last_active":"2024-09-14T07:11:07.010364+0000","last_peered":"2024-09-14T07:11:07.010364+0000","last_clean":"2024-09-14T07:11:07.010364+0000","last_became_active":"2024-09-14T07:09:11.349379+0000","last_became_peered":"2024-09-14T07:09:11.349379+0000","last_unstale":"2024-09-14T07:11:07.010364+0000","last_undegraded":"2024-09-14T07:11:07.010364+0000","last_fullsized":"2024-09-14T07:11:07.010364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:43:50.636292+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0052324850000000003,"stat_sum":{"num_bytes":71780368,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":11,"num_read":8359,"num_read_kb":47840,"num_write":367,"num_write_kb":16819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"29","length":"6"}]},{"pgid":"2.2","version":"19'2","reported_seq":349,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.830877+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:11:04.830877+0000","last_peered":"2024-09-14T07:11:04.830877+0000","last_clean":"2024-09-14T07:11:04.830877+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:11:04.830877+0000","last_undegraded":"2024-09-14T07:11:04.830877+0000","last_fullsized":"2024-09-14T07:11:04.830877+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"170'402","reported_seq":5821,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.828925+0000","last_change":"2024-09-14T07:11:03.945579+0000","last_active":"2024-09-14T07:11:04.828925+0000","last_peered":"2024-09-14T07:11:04.828925+0000","last_clean":"2024-09-14T07:11:04.828925+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:11:04.828925+0000","last_undegraded":"2024-09-14T07:11:04.828925+0000","last_fullsized":"2024-09-14T07:11:04.828925+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:57:36.813891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055297599999999999,"stat_sum":{"num_bytes":34046419,"num_objects":48,"num_object_clones":9,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":12,"num_read":7171,"num_read_kb":23431,"num_write":330,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.828899+0000","last_change":"2024-09-14T07:10:52.793359+0000","last_active":"2024-09-14T07:11:04.828899+0000","last_peered":"2024-09-14T07:11:04.828899+0000","last_clean":"2024-09-14T07:11:04.828899+0000","last_became_active":"2024-09-14T07:10:52.792887+0000","last_became_peered":"2024-09-14T07:10:52.792887+0000","last_unstale":"2024-09-14T07:11:04.828899+0000","last_undegraded":"2024-09-14T07:11:04.828899+0000","last_fullsized":"2024-09-14T07:11:04.828899+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:11.029506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"170'410","reported_seq":12640,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.808227+0000","last_change":"2024-09-14T07:11:03.815566+0000","last_active":"2024-09-14T07:11:04.808227+0000","last_peered":"2024-09-14T07:11:04.808227+0000","last_clean":"2024-09-14T07:11:04.808227+0000","last_became_active":"2024-09-14T07:10:41.121213+0000","last_became_peered":"2024-09-14T07:10:41.121213+0000","last_unstale":"2024-09-14T07:11:04.808227+0000","last_undegraded":"2024-09-14T07:11:04.808227+0000","last_fullsized":"2024-09-14T07:11:04.808227+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:05:08.638092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001925183,"stat_sum":{"num_bytes":31709737,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":18,"num_read":12588,"num_read_kb":19124,"num_write":337,"num_write_kb":7923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":339,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810399+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:11:04.810399+0000","last_peered":"2024-09-14T07:11:04.810399+0000","last_clean":"2024-09-14T07:11:04.810399+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:11:04.810399+0000","last_undegraded":"2024-09-14T07:11:04.810399+0000","last_fullsized":"2024-09-14T07:11:04.810399+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"170'368","reported_seq":7326,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.829036+0000","last_change":"2024-09-14T07:11:03.946808+0000","last_active":"2024-09-14T07:11:04.829036+0000","last_peered":"2024-09-14T07:11:04.829036+0000","last_clean":"2024-09-14T07:11:04.829036+0000","last_became_active":"2024-09-14T07:10:56.228666+0000","last_became_peered":"2024-09-14T07:10:56.228666+0000","last_unstale":"2024-09-14T07:11:04.829036+0000","last_undegraded":"2024-09-14T07:11:04.829036+0000","last_fullsized":"2024-09-14T07:11:04.829036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:51:04.703639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066396149999999998,"stat_sum":{"num_bytes":55405600,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":12,"num_read":6433,"num_read_kb":16009,"num_write":326,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":3508359,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"2.3","version":"17'1","reported_seq":348,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.831125+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:11:04.831125+0000","last_peered":"2024-09-14T07:11:04.831125+0000","last_clean":"2024-09-14T07:11:04.831125+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:11:04.831125+0000","last_undegraded":"2024-09-14T07:11:04.831125+0000","last_fullsized":"2024-09-14T07:11:04.831125+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":442,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810421+0000","last_change":"2024-09-14T07:09:11.418617+0000","last_active":"2024-09-14T07:11:04.810421+0000","last_peered":"2024-09-14T07:11:04.810421+0000","last_clean":"2024-09-14T07:11:04.810421+0000","last_became_active":"2024-09-14T07:09:11.350590+0000","last_became_peered":"2024-09-14T07:09:11.350590+0000","last_unstale":"2024-09-14T07:11:04.810421+0000","last_undegraded":"2024-09-14T07:11:04.810421+0000","last_fullsized":"2024-09-14T07:11:04.810421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:17.794780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"169'463","reported_seq":9730,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.828881+0000","last_change":"2024-09-14T07:11:03.944232+0000","last_active":"2024-09-14T07:11:04.828881+0000","last_peered":"2024-09-14T07:11:04.828881+0000","last_clean":"2024-09-14T07:11:04.828881+0000","last_became_active":"2024-09-14T07:10:58.788140+0000","last_became_peered":"2024-09-14T07:10:58.788140+0000","last_unstale":"2024-09-14T07:11:04.828881+0000","last_undegraded":"2024-09-14T07:11:04.828881+0000","last_fullsized":"2024-09-14T07:11:04.828881+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:36:50.166010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043375769999999996,"stat_sum":{"num_bytes":150553684,"num_objects":60,"num_object_clones":10,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":14,"num_read":9362,"num_read_kb":68884,"num_write":638,"num_write_kb":27467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.9","version":"174'641","reported_seq":7864,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T07:11:07.064455+0000","last_change":"2024-09-14T07:11:06.840118+0000","last_active":"2024-09-14T07:11:07.064455+0000","last_peered":"2024-09-14T07:11:07.064455+0000","last_clean":"2024-09-14T07:11:07.064455+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:11:07.064455+0000","last_undegraded":"2024-09-14T07:11:07.064455+0000","last_fullsized":"2024-09-14T07:11:07.064455+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:43:20.982879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001344654,"stat_sum":{"num_bytes":85413819,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":22,"num_read":9847,"num_read_kb":47844,"num_write":686,"num_write_kb":17456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"29","length":"6"}]},{"pgid":"3.a","version":"170'363","reported_seq":7780,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810952+0000","last_change":"2024-09-14T07:11:03.812274+0000","last_active":"2024-09-14T07:11:04.810952+0000","last_peered":"2024-09-14T07:11:04.810952+0000","last_clean":"2024-09-14T07:11:04.810952+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:11:04.810952+0000","last_undegraded":"2024-09-14T07:11:04.810952+0000","last_fullsized":"2024-09-14T07:11:04.810952+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:22:47.425407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049230979999999999,"stat_sum":{"num_bytes":36181044,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":7600,"num_read_kb":35747,"num_write":362,"num_write_kb":4780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.b","version":"171'914","reported_seq":7597,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.829093+0000","last_change":"2024-09-14T07:11:03.945321+0000","last_active":"2024-09-14T07:11:04.829093+0000","last_peered":"2024-09-14T07:11:04.829093+0000","last_clean":"2024-09-14T07:11:04.829093+0000","last_became_active":"2024-09-14T07:10:58.784908+0000","last_became_peered":"2024-09-14T07:10:58.784908+0000","last_unstale":"2024-09-14T07:11:04.829093+0000","last_undegraded":"2024-09-14T07:11:04.829093+0000","last_fullsized":"2024-09-14T07:11:04.829093+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":914,"log_dups_size":0,"ondisk_log_size":914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:34.571316+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055702440000000002,"stat_sum":{"num_bytes":71411531,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":9909,"num_read_kb":41316,"num_write":1291,"num_write_kb":13948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":231,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.c","version":"170'467","reported_seq":14931,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.831210+0000","last_change":"2024-09-14T07:11:03.814035+0000","last_active":"2024-09-14T07:11:04.831210+0000","last_peered":"2024-09-14T07:11:04.831210+0000","last_clean":"2024-09-14T07:11:04.831210+0000","last_became_active":"2024-09-14T07:09:13.778236+0000","last_became_peered":"2024-09-14T07:09:13.778236+0000","last_unstale":"2024-09-14T07:11:04.831210+0000","last_undegraded":"2024-09-14T07:11:04.831210+0000","last_fullsized":"2024-09-14T07:11:04.831210+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:10.126769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003088655,"stat_sum":{"num_bytes":55021603,"num_objects":41,"num_object_clones":5,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":14,"num_read":16150,"num_read_kb":37704,"num_write":399,"num_write_kb":7961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.d","version":"168'294","reported_seq":3812,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810705+0000","last_change":"2024-09-14T07:11:03.811028+0000","last_active":"2024-09-14T07:11:04.810705+0000","last_peered":"2024-09-14T07:11:04.810705+0000","last_clean":"2024-09-14T07:11:04.810705+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:11:04.810705+0000","last_undegraded":"2024-09-14T07:11:04.810705+0000","last_fullsized":"2024-09-14T07:11:04.810705+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:44.470154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037192919999999999,"stat_sum":{"num_bytes":32623828,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":16,"num_read":3330,"num_read_kb":19004,"num_write":293,"num_write_kb":4899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.e","version":"170'388","reported_seq":17066,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-14T07:11:06.838365+0000","last_change":"2024-09-14T07:11:06.838912+0000","last_active":"2024-09-14T07:11:06.838365+0000","last_peered":"2024-09-14T07:11:06.838365+0000","last_clean":"2024-09-14T07:11:06.838365+0000","last_became_active":"2024-09-14T07:10:57.226101+0000","last_became_peered":"2024-09-14T07:10:57.226101+0000","last_unstale":"2024-09-14T07:11:06.838365+0000","last_undegraded":"2024-09-14T07:11:06.838365+0000","last_fullsized":"2024-09-14T07:11:06.838365+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:11:50.150960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012820489999999999,"stat_sum":{"num_bytes":44501612,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":17343,"num_read_kb":39332,"num_write":391,"num_write_kb":5873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":7468264,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"29","length":"6"}]},{"pgid":"3.f","version":"171'400","reported_seq":8086,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.831633+0000","last_change":"2024-09-14T07:11:03.815472+0000","last_active":"2024-09-14T07:11:04.831633+0000","last_peered":"2024-09-14T07:11:04.831633+0000","last_clean":"2024-09-14T07:11:04.831633+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:11:04.831633+0000","last_undegraded":"2024-09-14T07:11:04.831633+0000","last_fullsized":"2024-09-14T07:11:04.831633+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:13.671493+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0043286339999999996,"stat_sum":{"num_bytes":32671897,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":8186,"num_read_kb":14036,"num_write":545,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.10","version":"170'428","reported_seq":7894,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.808101+0000","last_change":"2024-09-14T07:11:03.816872+0000","last_active":"2024-09-14T07:11:04.808101+0000","last_peered":"2024-09-14T07:11:04.808101+0000","last_clean":"2024-09-14T07:11:04.808101+0000","last_became_active":"2024-09-14T07:10:57.780512+0000","last_became_peered":"2024-09-14T07:10:57.780512+0000","last_unstale":"2024-09-14T07:11:04.808101+0000","last_undegraded":"2024-09-14T07:11:04.808101+0000","last_fullsized":"2024-09-14T07:11:04.808101+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:21.858299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031745509999999998,"stat_sum":{"num_bytes":56429858,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":7113,"num_read_kb":28845,"num_write":639,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":5309636,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.11","version":"170'473","reported_seq":8943,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810402+0000","last_change":"2024-09-14T07:11:03.808663+0000","last_active":"2024-09-14T07:11:04.810402+0000","last_peered":"2024-09-14T07:11:04.810402+0000","last_clean":"2024-09-14T07:11:04.810402+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:11:04.810402+0000","last_undegraded":"2024-09-14T07:11:04.810402+0000","last_fullsized":"2024-09-14T07:11:04.810402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:53:47.617751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872454,"stat_sum":{"num_bytes":61137551,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":13,"num_read":10109,"num_read_kb":24098,"num_write":373,"num_write_kb":9169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.12","version":"170'455","reported_seq":6638,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810648+0000","last_change":"2024-09-14T07:11:03.809700+0000","last_active":"2024-09-14T07:11:04.810648+0000","last_peered":"2024-09-14T07:11:04.810648+0000","last_clean":"2024-09-14T07:11:04.810648+0000","last_became_active":"2024-09-14T07:09:11.351724+0000","last_became_peered":"2024-09-14T07:09:11.351724+0000","last_unstale":"2024-09-14T07:11:04.810648+0000","last_undegraded":"2024-09-14T07:11:04.810648+0000","last_fullsized":"2024-09-14T07:11:04.810648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:01:37.493374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027793470000000002,"stat_sum":{"num_bytes":58990659,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":7176,"num_read_kb":22363,"num_write":364,"num_write_kb":7503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]},{"pgid":"3.13","version":"171'314","reported_seq":4536,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-14T07:11:04.810540+0000","last_change":"2024-09-14T07:11:03.811795+0000","last_active":"2024-09-14T07:11:04.810540+0000","last_peered":"2024-09-14T07:11:04.810540+0000","last_clean":"2024-09-14T07:11:04.810540+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:11:04.810540+0000","last_undegraded":"2024-09-14T07:11:04.810540+0000","last_fullsized":"2024-09-14T07:11:04.810540+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:09.801907+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0048898639999999998,"stat_sum":{"num_bytes":68165800,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":5699,"num_read_kb":45047,"num_write":317,"num_write_kb":5487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1784021626,"num_objects":1482,"num_object_clones":263,"num_object_copies":2964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1482,"num_whiteouts":460,"num_read":332109,"num_read_kb":1108332,"num_write":19969,"num_write_kb":329410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3271,"num_bytes_recovered":252991338,"num_keys_recovered":643,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45375488,"data_stored":391901853,"data_compressed":18899808,"data_compressed_allocated":36483072,"data_compressed_original":386464768,"omap_allocated":0,"internal_metadata":0},"log_size":16705,"ondisk_log_size":16705,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82224,"kb_used_data":10776,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94289616,"statfs":{"total":96636764160,"available":96552566784,"internally_reserved":0,"allocated":11034624,"data_stored":71666818,"data_compressed":3906786,"data_compressed_allocated":7229440,"data_compressed_original":69779456,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607599,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62936,"kb_used_data":5380,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94308904,"statfs":{"total":96636764160,"available":96572317696,"internally_reserved":0,"allocated":5509120,"data_stored":24318993,"data_compressed":1679350,"data_compressed_allocated":2678784,"data_compressed_original":23021568,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607598,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73212,"kb_used_data":10080,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94298628,"statfs":{"total":96636764160,"available":96561795072,"internally_reserved":0,"allocated":10321920,"data_stored":74154621,"data_compressed":3418721,"data_compressed_allocated":6651904,"data_compressed_original":72323072,"omap_allocated":1593,"internal_metadata":64616903},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607600,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67904,"kb_used_data":10340,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94303936,"statfs":{"total":96636764160,"available":96567230464,"internally_reserved":0,"allocated":10588160,"data_stored":80228224,"data_compressed":3514741,"data_compressed_allocated":6873088,"data_compressed_original":78354944,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607600,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67660,"kb_used_data":10104,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304180,"statfs":{"total":96636764160,"available":96567480320,"internally_reserved":0,"allocated":10346496,"data_stored":78913262,"data_compressed":3352610,"data_compressed_allocated":6668288,"data_compressed_original":77091328,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607600,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45176,"kb_used_data":9816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326664,"statfs":{"total":96636764160,"available":96590503936,"internally_reserved":0,"allocated":10051584,"data_stored":68866996,"data_compressed":3040550,"data_compressed_allocated":6455296,"data_compressed_original":67057664,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7950336,"data_stored":67432312,"data_compressed":3033658,"data_compressed_allocated":6418432,"data_compressed_original":66467840,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8294400,"data_stored":78073946,"data_compressed":3352610,"data_compressed_allocated":6668288,"data_compressed_original":77091328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8540160,"data_stored":79388927,"data_compressed":3514741,"data_compressed_allocated":6873088,"data_compressed_original":78354944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8204288,"data_stored":72719937,"data_compressed":3412663,"data_compressed_allocated":6615040,"data_compressed_original":71749632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8941568,"data_stored":70812073,"data_compressed":3906786,"data_compressed_allocated":7229440,"data_compressed_original":69779456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3444736,"data_stored":23474658,"data_compressed":1679350,"data_compressed_allocated":2678784,"data_compressed_original":23021568,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:11:09.581 INFO:tasks.thrashosds.thrasher:Setting 3.1a to [4, 0] 2024-09-14T07:11:09.581 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1a', '4', '0'] 2024-09-14T07:11:09.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1a 4 0 2024-09-14T07:11:09.637 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:09.637 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:09.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:09.671 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:09.670+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:09.761 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:09.758+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:09.820 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:09.820 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:09.834 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:09.859 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:11:09.876 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:09.874+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:09.962 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:09.958+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:10.016 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:10.016 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:10.036 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:10.079 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:10.078+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:10.164 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:10.162+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:10.168 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:10.168 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:10.183 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:10.268 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:10.266+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:10.348 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:10.348 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:10.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:10.404 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:10.402+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:10.468 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:10.466+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:10.500 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:10.500 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:10.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:10.568 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:10.566+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:10.669 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:10.666+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:10.686 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:10.687 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:10.687 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:10.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:10.791 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:10.790+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:10.853 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:10.854 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:10.863 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.1a pg_upmap mapping to [4,0] 2024-09-14T07:11:10.870 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:10.866+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:10.888 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:10.888 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1903 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5665280~36352,write 851968~31232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 173.60966311799999, 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085906362999999999, 2024-09-14T07:11:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1903 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288741+0000", 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.288742+0000", 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4e-07 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306966+0000", 2024-09-14T07:11:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018223746999999998 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.306991+0000", 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5519000000000001e-05 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307002+0000", 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0845999999999999e-05 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307070+0000", 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7751999999999998e-05 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.307203+0000", 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013295399999999999 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.308263+0000", 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001059817 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.373421+0000", 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065158524999999995 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374517+0000", 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001095688 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374580+0000", 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3551000000000002e-05 2024-09-14T07:11:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.374647+0000", 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7024000000000001e-05 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1904 3.b e43/21 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 43'148, mlcod=42'145)", 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 173.58015548700001, 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071452262000000002, 2024-09-14T07:11:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318249+0000", 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.318250+0000", 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.333e-06 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321701+0000", 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034506210000000001 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321723+0000", 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2157000000000001e-05 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321733+0000", 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0226e-05 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321802+0000", 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9166999999999997e-05 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.321829+0000", 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7229999999999998e-05 2024-09-14T07:11:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389538+0000", 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067708483 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.389701+0000", 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016304500000000001 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4803.0:1906 3.15 e43/21 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 43'70, mlcod=42'68)", 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 173.571508128, 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067471774999999998, 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326896+0000", 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.326897+0000", 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3740000000000001e-06 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339114+0000", 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012216216 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339138+0000", 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4476000000000001e-05 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339151+0000", 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3337e-05 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339267+0000", 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011519300000000001 2024-09-14T07:11:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.339318+0000", 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1638000000000003e-05 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394313+0000", 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05499457 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.394368+0000", 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4970999999999999e-05 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:675 3.18 e46/25 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head v 46'64, mlcod=42'61)", 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 168.905415788, 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.11721798, 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992988+0000", 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.992991+0000", 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439999999999998e-06 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005404+0000", 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012413222 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005458+0000", 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3921000000000001e-05 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005484+0000", 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6234999999999999e-05 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005644+0000", 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016038599999999999 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.005709+0000", 2024-09-14T07:11:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4127000000000005e-05 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110158+0000", 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.104448923 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110206+0000", 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8721999999999999e-05 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:831 3.10 e46/21 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head v 46'40, mlcod=39'38)", 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 168.738686562, 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.079354918999999996, 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:11:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159718+0000", 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.159719+0000", 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4339999999999999e-06 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171571+0000", 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011851970999999999 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171596+0000", 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069e-05 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171607+0000", 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1411e-05 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171696+0000", 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8247999999999999e-05 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.171721+0000", 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5732e-05 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239010+0000", 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.067288419000000002 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.239072+0000", 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2634999999999999e-05 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:3057 3.1b 3:d9c7c761:::rbd_data.1161ff1af5c4.0000000000000002:head [read 29986816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e50)", 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 164.80568555400001, 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14897993700000001, 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3057 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092719+0000", 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092721+0000", 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092730+0000", 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8380000000000008e-06 2024-09-14T07:11:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092754+0000", 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4439000000000001e-05 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092766+0000", 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1387e-05 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092872+0000", 2024-09-14T07:11:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106174 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.092887+0000", 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5012e-05 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241204+0000", 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.148316901 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241458+0000", 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000253942 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:26.241698+0000", 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024087699999999999 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1416 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.05024238199999, 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040632382000000002, 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1416 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848162+0000", 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848164+0000", 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9259999999999999e-06 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848172+0000", 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848175+0000", 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:11:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888544+0000", 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040369156000000003 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888583+0000", 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888794+0000", 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000210707 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1417 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 485376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.05022446, 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040926006000000001, 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1417 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848180+0000", 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848181+0000", 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848186+0000", 2024-09-14T07:11:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848188+0000", 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888821+0000", 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040633157000000003 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.888864+0000", 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2166000000000003e-05 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889106+0000", 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000242007 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1418 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.050211691, 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041177634999999997, 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1418 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848192+0000", 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848193+0000", 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:11:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848194+0000", 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848197+0000", 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.083e-06 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848199+0000", 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889121+0000", 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040921851000000002 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889159+0000", 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7267000000000001e-05 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889370+0000", 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021147900000000001 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1419 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 507904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.050201515, 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041489012999999998, 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1419 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848203+0000", 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848204+0000", 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2529999999999999e-06 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848207+0000", 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9919999999999999e-06 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848210+0000", 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889399+0000", 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041189152 2024-09-14T07:11:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889438+0000", 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9313000000000001e-05 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889692+0000", 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025350900000000002 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1420 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 516096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.05019132, 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041742253, 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1420 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848213+0000", 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-09-14T07:11:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848214+0000", 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1260000000000001e-06 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848217+0000", 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848220+0000", 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5840000000000002e-06 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889708+0000", 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041487905999999998 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889746+0000", 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8387999999999999e-05 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889955+0000", 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020907900000000001 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1421 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 530432~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 114.05018130000001, 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041857289999999998, 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1421 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848223+0000", 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848224+0000", 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848227+0000", 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1070000000000001e-06 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.848230+0000", 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.889981+0000", 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041750866999999997 2024-09-14T07:11:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890021+0000", 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0918999999999999e-05 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.890080+0000", 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8576999999999998e-05 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5436.0:1646 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 921600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 113.99668018600001, 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040613986999999997, 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5436", 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4019460112", 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1646 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901724+0000", 2024-09-14T07:11:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901727+0000", 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.638e-06 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901734+0000", 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3470000000000001e-06 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.901739+0000", 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9649999999999999e-06 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942253+0000", 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040513228999999998 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942289+0000", 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6893000000000001e-05 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:16.942338+0000", 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8469e-05 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2704 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 210944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.836372674000003, 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041076901999999998, 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2704 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062031+0000", 2024-09-14T07:11:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062032+0000", 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062036+0000", 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5099999999999999e-06 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062045+0000", 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062053+0000", 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5799999999999992e-06 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102274+0000", 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040220524000000001 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.102386+0000", 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011198 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103108+0000", 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072268899999999999 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2706 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 224256~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.836341134999998, 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042110739000000001, 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2706 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062063+0000", 2024-09-14T07:11:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062064+0000", 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999997e-07 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062067+0000", 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062076+0000", 2024-09-14T07:11:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2099999999999999e-06 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062085+0000", 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103185+0000", 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041100638000000002 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.103372+0000", 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018678499999999999 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104174+0000", 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000801733 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2708 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 238592~5120] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:11:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.836309822999993, 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043244117999999998, 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2708 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062094+0000", 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062095+0000", 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2699999999999999e-07 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062107+0000", 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2116000000000001e-05 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062117+0000", 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6500000000000008e-06 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062124+0000", 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0670000000000003e-06 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104303+0000", 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042179571999999999 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.104551+0000", 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024771800000000002 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105338+0000", 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078736799999999999 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:2709 3.1b 3:dd56a889:::rbd_data.1161317ab2a.0000000000000024:1e [read 252928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.836269845999993, 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044159137000000001, 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2709 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062134+0000", 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062135+0000", 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062138+0000", 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062147+0000", 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9299999999999992e-06 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.062154+0000", 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9160000000000004e-06 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105455+0000", 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043301012999999999 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.105656+0000", 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020108100000000001 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.106293+0000", 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000637091 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5548.0:3111 3.1b 3.fcb7041b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 82.826649462999995, 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040579466000000002, 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5548", 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1700367335", 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3111 2024-09-14T07:11:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071755+0000", 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071756+0000", 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071761+0000", 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1730000000000003e-06 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071777+0000", 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6424000000000001e-05 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.071790+0000", 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2660000000000001e-05 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112311+0000", 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040521319 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:48.112334+0000", 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2860000000000001e-05 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.5713.0:490 3.1 e124/88 3:8109c77f:::rbd_data.11618d2dcb2b.0000000000000006:head v 124'192, mlcod=118'189)", 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.800907663999993, 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.061557063000000002, 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097496+0000", 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.097499+0000", 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107097+0000", 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0095975739999999993 2024-09-14T07:11:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107128+0000", 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1488000000000001e-05 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107143+0000", 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107270+0000", 2024-09-14T07:11:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000126549 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.107311+0000", 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1193999999999999e-05 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.158975+0000", 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051664531 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.159053+0000", 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7992999999999997e-05 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22321 3.10 3:0e955a9d:::rbd_data.1161b7591419.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e164)", 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.630066282, 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.52034672699999995, 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22321 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268338+0000", 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268340+0000", 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.784e-06 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268346+0000", 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268364+0000", 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7825999999999998e-05 2024-09-14T07:11:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268373+0000", 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.577e-06 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268447+0000", 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3978000000000002e-05 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.268463+0000", 2024-09-14T07:11:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5478999999999999e-05 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.781808+0000", 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.51334471599999998 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.782098+0000", 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00029031499999999997 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.782282+0000", 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018428300000000001 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788372+0000", 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0060903279999999999 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788543+0000", 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017061499999999999 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788622+0000", 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9059000000000003e-05 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.788685+0000", 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2395000000000001e-05 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:10.992 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:10.993 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:10.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:10.995 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:10.996 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:10.996 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:10.997 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:10.990+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:11.095 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:11.090+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:11.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:11.231 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:11.230+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:11.297 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:11.294+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:11.312 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:11.313 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:11.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:11.398 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:11.394+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:11.499 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:11.494+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:11.501 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 169.820004168, 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428998533, 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428966258, 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428908155, 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42886725300001, 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42880544499999, 2024-09-14T07:11:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428771129, 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42875245800001, 2024-09-14T07:11:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42872842200001, 2024-09-14T07:11:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428708388, 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428693553, 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42867892699999, 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428664426, 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428646671, 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42863177700001, 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42861766599999, 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.42859573699999, 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 124.428574426, 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 120.141223525, 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.639555136, 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:11.594 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:11.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:11.599 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:11.598+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:11.700 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:11.698+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:11.758 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:11.758 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:11.758 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:11.758 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:11.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:11.801 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:11.798+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:11.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:11:11.923 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:11.923 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:11.924 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:11.924 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:11.922+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:11.939 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:12.031 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:12.030+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:12.103 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:12.098+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:12.105 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:12.105 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:12.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:12.105 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:12.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.105 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:12.105 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 174.802569439, 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:12.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 174.75757913300001, 2024-09-14T07:11:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 170.455646677, 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:12.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.122 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 170.13431894300001, 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:12.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:12.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.128 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.69719054399999, 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.697179886, 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:12.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:12.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.137 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.697164872, 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:12.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:12.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.697153957, 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:12.142 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:12.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:12.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.697142447, 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:12.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:12.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:12.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.69712990399999, 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:12.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.155 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.69028330800001, 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.690235527, 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.163 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.69022206, 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:12.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.690211088, 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.690200326, 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:12.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:12.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.690184272, 2024-09-14T07:11:12.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:12.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.69016770100001, 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:12.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.690155925, 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:12.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.69014524799999, 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:12.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:12.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.194 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.528968823, 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:12.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:12.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:12.201 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:12.201 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:12.202 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:12.203 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:12.202+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:12.304 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:12.302+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:12.333 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:12.334 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:12.334 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:12.334 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:12.349 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:12.405 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:12.402+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:12.468 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:12.469 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:12.469 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:12.469 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:12.469 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:12.482 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:12.528 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:12.526+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 220.305110536, 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:12.608 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:12.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:12.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 144.41111735199999, 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:12.615 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:12.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:12.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:12.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:12.623 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 144.41099747300001, 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:12.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:12.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:12.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:12.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 144.393772353, 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:12.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:12.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 121.721177966, 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.637 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:12.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.94703991199999, 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:12.643 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:12.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:12.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.94700621200001, 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:12.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:12.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.652 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.94698556900001, 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:12.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:12.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.94674035200001, 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:12.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:12.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:12.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:12.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.946708633, 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.665 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.94668687399999, 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:12.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:12.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.946654795, 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:12.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.674 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.94548358900001, 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:12.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.678 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.945460626, 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.679 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:12.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:12.683 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.94543933600001, 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:12.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:12.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:12.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 119.945419113, 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:12.688 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 42.817693009000003, 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:12.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:12.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:12.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.818447895999999, 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.698 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:12.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.818400504, 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.703 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:12.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:12.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:12.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 19.818358976999999, 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.709 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:12.713 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:12.714 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:12.606+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:12.715 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:12.716 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:12.714+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:12.783 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:11:12.847 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:12.846+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:12.886 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:12.886 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:12.886 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:12.886 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:12.900 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:12.943 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:12.942+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:12.998 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:13.018 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:13.018 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:13.018 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:13.018 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:13.018 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:13.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:13.047 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:13.046+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:13.120 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:13.118+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:13.153 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:13.153 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:13.169 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:13.169 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:13.185 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:13.186 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:13.203 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:13.203 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 175.90538540700001, 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 163.04590981999999, 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 163.04580753499999, 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.346755694, 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.861744133, 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.86171006399999, 2024-09-14T07:11:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.861666718, 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.861633195, 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.861585945, 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.861471697, 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.86136200200001, 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 126.86129585899999, 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.626220011000001, 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 61.339506575000001, 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:13.269 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.723491533000001, 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:13.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:13.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:13.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.723427833999999, 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:13.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:13.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:13.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.723384500000002, 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:13.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:13.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:13.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:13.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:13.287 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.723327621000003, 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:13.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.723261125999997, 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.293 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:13.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:13.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:13.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 57.720376788999999, 2024-09-14T07:11:13.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:13.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:13.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:13.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:13.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.306 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.306 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:13.306 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:13.306 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:13.307 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:13.307 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:13.308 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:13.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:13.309 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:13.306+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:13.408 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:13.406+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:13.497 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:13.497 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:13.508 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:13.506+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:13.511 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:13.609 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:13.606+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:13.693 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:13.694 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:13.709 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:13.752 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:13.750+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:13.810 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:13.806+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:13.875 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:13.875 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:13.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:13.943 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:13.942+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:14.012 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:14.010+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:14.029 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:14.029 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:14.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:14.113 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:14.110+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:14.214 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:14.210+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:14.217 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:14.217 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:14.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:14.314 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:14.310+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:14.400 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:14.400 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:14.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:14.415 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:14.414+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:14.517 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:14.514+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:14.541 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:14.541 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:14.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:14.618 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:14.614+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:14.682 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:14.682 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:14.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:14.743 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:14.742+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:14.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:11:14.816 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:14.816 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:14.819 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:14.818+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:14.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:14.927 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:14.926+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:15.022 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:15.018+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:15.026 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:15.026 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:15.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:15.131 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:15.130+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:15.218 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:15.218 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:15.223 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:15.222+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:15.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:15.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:15.363 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:15.362+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:15.401 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:15.401 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:15.416 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:15.464 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:15.462+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:15.471 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:15.488 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:15.506 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:15.506 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:15.525 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:15.522+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:15.528 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:15.528 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:15.549 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:15.549 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:15.549 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:15.552 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:15.552 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:15.563 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:15.574 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:15.574 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:15.656 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:15.654+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:15.728 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:15.726+0000 7f7d106e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25146) UID: 0 2024-09-14T07:11:15.728 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:15.728 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:15.728 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:15.728 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:15.729 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:15.742 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:15.828 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:15.826+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:15.890 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-14T07:11:15.890 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:11:15.890 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-14T07:11:15.890 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T07:11:15.891 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:11:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 223.61344119699999, 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:15.916 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:15.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:15.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:15.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.719448013, 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.923 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:15.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:15.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:15.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.71932813399999, 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:15.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:15.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.70210301399999, 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:15.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:15.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:15.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 125.029508627, 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:15.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.25537057299999, 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:15.950 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:15.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.255336873, 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:15.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:15.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.25531623000001, 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.25507101300001, 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.964 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:15.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.968 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.255039294, 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:15.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.25501753499999, 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.973 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:15.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:15.977 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.254985456, 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:15.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.982 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.25381425, 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:15.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:15.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.986 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.253791287, 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.987 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:15.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.25376999700001, 2024-09-14T07:11:15.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:15.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:15.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 123.253749774, 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.996 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:15.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:15.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:15.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.126023670000002, 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:16.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:16.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:16.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.005 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.126778557000002, 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.010 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.126731164999999, 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:16.011 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:16.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:16.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 23.126689637999998, 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:16.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.021 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:16.023 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:16.024 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:16.018+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:16.124 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:16.122+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:16.200 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:16.200 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:16.200 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:16.200 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:16.214 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:16.259 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:16.258+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:16.325 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:16.322+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:16.336 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:16.336 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:16.336 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:16.336 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:16.336 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:16.350 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:16.426 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:16.422+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:16.516 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:16.516 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:16.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:16.516 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:16.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 203.68316861400001, 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.517 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:16.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 203.68239426599999, 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.522 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.523 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:16.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:16.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 174.67121126800001, 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:16.527 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:16.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:16.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 174.37428306199999, 2024-09-14T07:11:16.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:16.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.139379238, 2024-09-14T07:11:16.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.543 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.13881744700001, 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.544 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:16.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.13832288, 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:16.548 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:16.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.552 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.137705461, 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.553 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:16.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:16.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:16.557 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.13727720200001, 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.558 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:16.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.13688133799999, 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:16.562 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:16.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.566 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.136289662, 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.567 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:16.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.13271491899999, 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:16.571 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.575 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.13267556599999, 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.576 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:16.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.580 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.132637786, 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:16.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:16.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 130.132601083, 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:16.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.589 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.543229148999998, 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:16.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.538556717000006, 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.594 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:16.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.538474225000002, 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:16.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:16.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.538313103999997, 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:16.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.607 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 68.538136195999996, 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:16.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:16.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:16.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.612 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:16.612 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:16.612 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:16.613 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:16.610+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:16.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:16.714 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:16.710+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:16.751 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:16.751 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:16.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:16.815 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:16.814+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:16.916 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:16.914+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:16.945 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:16.945 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:16.946 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:16.946 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:16.946 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:16.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:17.017 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:17.014+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:17.091 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 175.40960265300001, 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:17.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01859701800001, 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:11:17.096 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:11:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:17.100 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01856474300001, 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:17.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:17.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01850664, 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:17.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:17.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:17.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.018465738, 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:17.108 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01840393000001, 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01836961399999, 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:17.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:17.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.018350943, 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:17.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:17.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:17.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:17.124 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01832690699999, 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:17.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:17.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.018306873, 2024-09-14T07:11:17.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:17.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:17.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.018292038, 2024-09-14T07:11:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:17.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.018277412, 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:17.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01826291099999, 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:17.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01824515600001, 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.148 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.018230262, 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01821615099999, 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01819422200001, 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.160 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 130.01817291099999, 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.161 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 125.73082201, 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:17.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:17.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:17.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.229153620999998, 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:17.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:17.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:17.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:17.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.176 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.176 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.176 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:17.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:17.178 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:17.174+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:17.279 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:17.274+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:17.309 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:17.309 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:17.309 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:17.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:17.415 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:17.414+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:17.480 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:17.478+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:17.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:17.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:17.595 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:17.594+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:17.683 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 180.38147452000001, 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:17.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:17.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:17.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 180.336484214, 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:17.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:17.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 176.03455175799999, 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:17.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:17.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 175.713224024, 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:17.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:17.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:17.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.27609562500001, 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.711 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.276084967, 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:17.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.27606995299999, 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:17.721 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:17.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:17.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.276059038, 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:17.726 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:17.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.27604752799999, 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:17.731 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:17.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:17.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.276034985, 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:17.736 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.26918838899999, 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:17.741 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:17.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.26914060799999, 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.26912714100001, 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:17.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:17.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:17.756 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.269116169, 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:17.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:17.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.761 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.26910540700001, 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:17.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.269089353, 2024-09-14T07:11:17.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:17.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:17.772 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.26907278199999, 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.26906100599999, 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:17.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:17.782 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 131.26905032900001, 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:17.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.787 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.107873904000002, 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:17.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:17.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:17.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:17.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:17.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:17.795 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:17.795 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:17.678+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:17.797 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:17.797 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:17.798 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:17.799 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:17.800 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:17.798+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:17.806 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:17.826 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:17.831 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:17.831 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:17.901 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:17.898+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:17.940 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:17.940 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:17.956 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:18.002 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:17.998+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:18.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:11:18.078 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:18.078 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:18.094 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:18.136 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:18.134+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:18.203 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:18.202+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:18.261 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:18.261 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:18.275 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:18.305 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:18.302+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:18.399 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:18.399 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:18.406 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:18.402+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:18.413 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:18.506 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:18.502+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:18.581 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:18.581 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:18.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:18.640 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:18.638+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:18.708 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:18.706+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:18.763 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:18.763 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:18.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:18.820 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:18.818+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:18.909 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:18.905+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:18.910 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:11:18.952 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:18.952 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:18.967 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:19.012 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:19.010+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:19.096 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:19.096 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:19.110 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:19.152 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:19.150+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:19.213 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:19.209+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:19.277 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:19.277 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:19.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:19.313 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:19.310+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:19.414 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:19.410+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:19.425 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:19.425 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:19.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:19.515 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:19.514+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:19.559 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:19.560 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:19.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:19.615 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:19.614+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:19.705 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:19.705 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:19.716 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:19.714+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:19.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:19.817 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:19.814+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:19.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:19.844 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:19.845 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:19.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:19.918 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:19.914+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:19.970 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:20.007 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:20.008 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:20.019 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:20.014+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:20.041 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:20.042 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:20.058 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:20.058 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:20.058 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:20.068 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:20.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:20.092 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:20.112 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:20.113 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:20.116 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:20.117 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:20.120 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:20.118+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:20.196 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:20.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 182.89521018299999, 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:20.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:20.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 182.850219877, 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:20.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:20.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:20.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:20.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:20.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 178.548287421, 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:20.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 178.226959687, 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:20.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:20.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:20.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78983128799999, 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:20.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78982063000001, 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.228 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:20.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.789805616, 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.233 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:20.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78979470100001, 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:20.237 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:20.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.789783191, 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.246 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.789770648, 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.782924052, 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:20.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78287627099999, 2024-09-14T07:11:20.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.259 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78286280399999, 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:20.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.264 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78285183200001, 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:20.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:20.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:20.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78284106999999, 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.782825016, 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.782808445, 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78279666899999, 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.78278599199999, 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.621609566999993, 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:20.295 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:20.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:20.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:20.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:20.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:20.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:20.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.302 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.302 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:20.302 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:20.302 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:20.304 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:20.304 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:20.302+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:20.405 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:20.401+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:20.436 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:20.436 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:20.437 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:20.450 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:20.505 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:20.502+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:20.575 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:20.575 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:20.575 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:20.575 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:20.576 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:20.589 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:20.606 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:20.605+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:20.707 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:20.702+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:20.764 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 228.46138330799999, 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.765 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.771 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 152.56739012400001, 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:20.772 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:20.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:20.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:20.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:20.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 152.567270245, 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:20.780 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:20.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 152.550045125, 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.793 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 129.87745073799999, 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:20.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:20.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:20.799 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.103312684, 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:20.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:20.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.803 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10327898400001, 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:20.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10325834099999, 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:20.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:20.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.103013124, 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:20.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.816 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10298140500001, 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:20.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10295964599999, 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.821 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:20.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.825 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10292756699999, 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:20.829 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10175636100001, 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.830 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:20.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10173339799999, 2024-09-14T07:11:20.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:20.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.838 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10171210799999, 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:20.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:20.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:20.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.842 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 128.10169188500001, 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:20.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.973965780999997, 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:20.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.852 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.974720668, 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:20.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:20.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:20.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:20.857 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.974673276000001, 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:20.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:20.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 27.974631749, 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.863 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:20.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:20.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:20.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.868 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.868 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:20.868 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:20.868 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:20.869 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:20.870 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:20.866+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:20.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:11:20.970 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:20.966+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:21.047 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:24975 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=28b out=50b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e188)", 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:11:21.047586+0000", 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00028103999999999998, 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028891600000000001, 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:21.048 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 24975 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047586+0000", 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047586+0000", 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047588+0000", 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.03e-06 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047599+0000", 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0832999999999999e-05 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047627+0000", 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8079999999999999e-05 2024-09-14T07:11:21.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047648+0000", 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.128e-05 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047728+0000", 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-05 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047776+0000", 2024-09-14T07:11:21.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8599e-05 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:11:21.047875+0000", 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8703999999999996e-05 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-14T07:11:21.052 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:21.062 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:21.104 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:21.102+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:21.172 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:21.169+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:21.226 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:21.227 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:21.227 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:21.241 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:21.288 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:21.286+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:21.367 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 184.067422795, 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:21.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:21.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 171.20794720800001, 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.373 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 171.20784492300001, 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:21.378 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:21.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 170.50879308200001, 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.383 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:21.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:21.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.02378152099999, 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.388 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:21.392 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.02374745200001, 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:21.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:21.396 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.023704106, 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:21.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.02367058300001, 2024-09-14T07:11:21.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:21.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:21.404 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.02362333299999, 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:21.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:21.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.023509085, 2024-09-14T07:11:21.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:21.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:21.412 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.02339939000001, 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:21.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.416 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.02333324700001, 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:21.420 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.788257399000003, 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:21.425 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.501543963000003, 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:21.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:21.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.885528921000002, 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.432 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:21.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.885465221999993, 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:21.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:21.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:21.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.885421887999996, 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:21.443 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:21.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.885365008999997, 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:21.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:21.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.454 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.885298513999999, 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:21.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 65.882414177000001, 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:21.460 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:21.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:21.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:21.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:21.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.466 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:21.467 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:21.468 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:21.466+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:21.568 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:21.565+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:21.641 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:21.641 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:21.641 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:21.641 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:21.657 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:21.668 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:21.665+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:21.769 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:21.766+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:21.777 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:21.777 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:21.777 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:21.777 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:21.778 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:21.791 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:21.870 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:21.865+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:21.939 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 209.138169973, 2024-09-14T07:11:21.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:21.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.976 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 209.13739562500001, 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:21.981 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 180.126212627, 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:21.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:21.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 179.82928442100001, 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.987 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:21.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.594380597, 2024-09-14T07:11:21.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:21.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:21.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.997 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.593818806, 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:21.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:21.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:22.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.001 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.593324239, 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.005 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.59270681999999, 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:22.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:22.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:22.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.592278561, 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.59188269699999, 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.014 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:22.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.59129102099999, 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:22.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:22.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.022 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.58771627799999, 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:22.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:22.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.58767692500001, 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:22.027 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.031 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.587639145, 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.032 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.033 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.58760244199999, 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.036 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:22.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:22.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.998230508000006, 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:22.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:22.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.993558075999999, 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:22.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.993475583999995, 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.049 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.051 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.993314463000004, 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.053 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.993137555000004, 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:22.057 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:22.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:22.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:22.061 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:22.062 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:22.062 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T07:11:22.062 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-14T07:11:22.062 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:11:22.063 DEBUG:teuthology.orchestra.run.smithi005:> type ceph-objectstore-tool 2024-09-14T07:11:22.063 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:22.005+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:22.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:22.069 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T07:11:22.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-14T07:11:22.111 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:22.109+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:22.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:22.202 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:22.202 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:22.202 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:22.202 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:22.202 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:22.197+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:22.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:22.273 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:22.270+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:22.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:22.316 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:22.337 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:22.358 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:22.374 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:22.369+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:22.378 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:22.378 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:22.379 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:22.379 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:22.384 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:22.385 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:22.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:22.474 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:22.470+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:22.559 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:22.559 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:22.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:22.576 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:22.574+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:3.c 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:3.b 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:1.0 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:2.4 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:2.5 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:3.10 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:3.12 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:3.17 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:3.3 2024-09-14T07:11:22.658 INFO:teuthology.orchestra.run.smithi005.stdout:3.6 2024-09-14T07:11:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:3.15 2024-09-14T07:11:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:3.18 2024-09-14T07:11:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:3.1 2024-09-14T07:11:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:3.1a 2024-09-14T07:11:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:3.1b 2024-09-14T07:11:22.659 INFO:teuthology.orchestra.run.smithi005.stdout:3.1f 2024-09-14T07:11:22.676 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:22.674+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:22.693 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:22.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:22.765 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-14T07:11:22.776 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:22.773+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:22.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:22.891 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:22.889+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:22.978 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:22.974+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:22.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.1f --file /var/log/ceph/exp.3.1f.0 2024-09-14T07:11:23.022 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:23.022 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:23.037 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:23.079 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:23.077+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:23.169 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:23.170 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:23.180 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:23.178+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:23.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:23.281 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:23.278+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:23.354 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:23.354 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:23.368 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:23.382 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:23.377+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:23.482 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:23.478+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:23.537 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:23.537 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:23.552 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:23.596 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:23.594+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:23.598 INFO:teuthology.orchestra.run.smithi005.stderr:Exporting 3.1f info 3.1f( v 177'362 (0'0,177'362] local-lis/les=88/89 n=31 ec=21/21 lis/c=88/88 les/c/f=89/89/0 sis=88) 2024-09-14T07:11:23.602 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f882f4e5:::rbd_data.116170297dfe.0000000000000009:head# 2024-09-14T07:11:23.603 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f8928d1a:::rbd_data.1161a9b4f57.0000000000000036:head# 2024-09-14T07:11:23.603 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f8a9ac2c:::rbd_data.1161a6ea4953.00000000000000d1:1b# 2024-09-14T07:11:23.603 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f8a9ac2c:::rbd_data.1161a6ea4953.00000000000000d1:head# 2024-09-14T07:11:23.603 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:head# 2024-09-14T07:11:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f925b192:::rbd_data.1161ca0f0ebc.0000000000000050:head# 2024-09-14T07:11:23.612 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f9b02d91:::rbd_data.1161605482a1.0000000000000004:head# 2024-09-14T07:11:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f9b060bf:::rbd_data.1161ca0f0ebc.0000000000000043:head# 2024-09-14T07:11:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:f9bbddff:::rbd_data.1161ce824eb3.0000000000000043:head# 2024-09-14T07:11:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fa01a741:::rbd_data.1161a0b955f2.0000000000000040:head# 2024-09-14T07:11:23.619 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:b# 2024-09-14T07:11:23.649 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head# 2024-09-14T07:11:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fa52177d:::rbd_data.11611fdb64aa.0000000000000020:head# 2024-09-14T07:11:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fa654f18:::rbd_id.image_client.0:head# 2024-09-14T07:11:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fa782143:::rbd_data.1161a6fd7139.0000000000000034:23# 2024-09-14T07:11:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fa782143:::rbd_data.1161a6fd7139.0000000000000034:head# 2024-09-14T07:11:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fb633a10:::rbd_data.1161a9b4f57.0000000000000081:head# 2024-09-14T07:11:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fbac3711:::rbd_data.1161a9b4f57.000000000000005d:head# 2024-09-14T07:11:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fbbfc85c:::rbd_data.1161a6fd7139.0000000000000007:head# 2024-09-14T07:11:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fbef71f2:::rbd_data.116155d7d34e.0000000000000009:head# 2024-09-14T07:11:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fc2fb453:::rbd_data.116148ee4e32.0000000000000031:head# 2024-09-14T07:11:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fcb62c1a:::rbd_data.116148ee4e32.0000000000000009:head# 2024-09-14T07:11:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fcd26671:::rbd_data.1161a9b4f57.0000000000000045:head# 2024-09-14T07:11:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fd7add64:::rbd_data.1161dbabc1ae.00000000000000d4:head# 2024-09-14T07:11:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fdbc7aa9:::rbd_data.116167e24a2d.000000000000000f:head# 2024-09-14T07:11:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fdc075a7:::rbd_data.1161ce824eb3.0000000000000040:head# 2024-09-14T07:11:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fddcb068:::rbd_data.1161a9b4f57.0000000000000060:head# 2024-09-14T07:11:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:fe62e861:::rbd_data.1161ec00b0f6.000000000000003c:head# 2024-09-14T07:11:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ff7206e7:::rbd_data.1161ca0f0ebc.0000000000000051:head# 2024-09-14T07:11:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:10# 2024-09-14T07:11:23.685 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:23.681+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head# 2024-09-14T07:11:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:Export successful 2024-09-14T07:11:23.720 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:23.720 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:23.734 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:23.785 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:23.785+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:23.870 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:23.916 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:23.914+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:23.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:11:23.987 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:23.982+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:23.997 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:23.997 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:24.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.1f 2024-09-14T07:11:24.012 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:24.103 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:24.102+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:24.189 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:24.186+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:24.193 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:24.193 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:24.207 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:24.296 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:24.294+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:24.372 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:24.372 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:24.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:24.387 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:24.428 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:24.426+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:24.492 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:24.489+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:24.539 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:24.539 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:24.550 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:24.550 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:24.559 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:24.559 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:24.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:24.580 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:24.581 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:24.593 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:24.590+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:24.599 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:24.599 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:24.634 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:24.638 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:24.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:24.682 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:24.682 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:24.694 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:24.690+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:24.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:24.794 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:24.789+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:24.873 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:24.873 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:24.873 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:24.873 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:24.873 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:24.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:24.931 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:24.929+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:24.946 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:11:25.019 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:25.017+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:25.072 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 183.390650631, 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:25.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:25.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.999644996, 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.077 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:11:25.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:11:25.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99961272100001, 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.081 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:25.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:25.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99955461799999, 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.085 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.999513716, 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.089 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:25.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.999451908, 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99941759199999, 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99939892099999, 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:25.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:25.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99937488500001, 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:25.105 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:25.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:25.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99935485099999, 2024-09-14T07:11:25.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:25.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:25.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:25.113 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99934001599999, 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99932539, 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.121 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99931088899999, 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:25.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.999293134, 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:25.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.129 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99927824, 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.133 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99926412900001, 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:25.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.9992422, 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.138 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:25.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 137.99922088899999, 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:25.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 133.71186998799999, 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:25.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:25.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:25.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.210201599000001, 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:25.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:25.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:25.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:25.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.157 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.157 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.157 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:25.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:25.159 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:25.153+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:25.259 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:25.258+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:25.325 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:25.325 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:25.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:25.360 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:25.358+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:25.462 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:25.458+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:25.462 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:25.462 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:25.462 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:25.462 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:25.462 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:25.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:25.563 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:25.558+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:25.604 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:25.604 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 188.30335743200001, 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:25.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:25.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:25.610 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 188.258367126, 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:25.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 183.95643466999999, 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:25.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:25.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:25.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:25.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 183.635106936, 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:25.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:25.627 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19797853700001, 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:25.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:25.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.197967879, 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:25.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:25.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19795286499999, 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19794195, 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:25.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:25.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19793043999999, 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.197917897, 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:25.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19107130099999, 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:25.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.657 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19102351999999, 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:25.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19101005300001, 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:25.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.190999081, 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:25.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19098831900001, 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.190972265, 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:25.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:25.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.190955694, 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:25.683 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19094391799999, 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:25.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:25.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.19093324100001, 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:25.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:25.692 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 83.029756816000003, 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:25.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:25.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:25.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:25.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:25.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.699 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:25.699 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:25.699 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:25.700 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:25.701 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:25.698+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:25.801 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:25.798+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:25.831 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:25.831 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:25.831 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:25.831 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:25.846 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:25.932 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:25.930+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:26.003 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:26.001+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:26.025 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:26.025 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:26.025 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:26.025 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:26.025 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:26.042 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:26.066 INFO:teuthology.orchestra.run.smithi005.stdout: marking collection for removal 2024-09-14T07:11:26.066 INFO:teuthology.orchestra.run.smithi005.stdout:setting '_remove' omap key 2024-09-14T07:11:26.066 INFO:teuthology.orchestra.run.smithi005.stdout:finish_remove_pgs 3.1f_head removing 3.1f 2024-09-14T07:11:26.071 INFO:teuthology.orchestra.run.smithi005.stdout:Remove successful 2024-09-14T07:11:26.127 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:26.126+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:26.206 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:26.201+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:26.221 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:26.221 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 233.91945247800001, 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:26.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:26.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:26.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:26.228 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 158.025459294, 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:26.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:26.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:26.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 158.02533941499999, 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.237 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:26.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:26.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.244 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 158.00811429500001, 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:26.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:26.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 135.33551990800001, 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.250 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:26.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.255 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.56138185399999, 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.256 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.561348154, 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.260 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.264 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.561327511, 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:26.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.56108229399999, 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:26.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.561050575, 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:26.273 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:26.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:26.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.561028816, 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:26.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:26.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:26.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.56099673700001, 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:26.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:26.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.559825531, 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:26.286 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:26.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:26.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.55980256800001, 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:26.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:26.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.559781278, 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.295 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:26.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:26.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:26.299 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 133.559761055, 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:26.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:26.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 56.432034950999999, 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:26.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:26.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.432789837999998, 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:26.310 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:26.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.432742445999999, 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.315 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:26.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.432700918999998, 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:26.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:26.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.325 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.325 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.325 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:26.326 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:26.327 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:26.325+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:26.427 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:26.421+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:26.496 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:26.497 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:26.497 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:26.497 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:26.511 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:26.556 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:26.554+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:26.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1f.0 2024-09-14T07:11:26.630 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:26.626+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:26.631 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:26.631 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:26.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:26.645 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:26.736 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:26.734+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:26.801 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 189.51640040999999, 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 176.656924823, 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 176.656822538, 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 175.957770697, 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.47275913600001, 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.472725067, 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.47268172099999, 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.472648198, 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.47260094800001, 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.47248669999999, 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.472377005, 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 140.472310862, 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.237235014000007, 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 74.950521578000007, 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 71.334506536000006, 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:26.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:26.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 71.334442836999997, 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:26.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:26.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 71.334399503, 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:26.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:26.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:26.898 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 71.334342624000001, 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:26.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 71.334276129000003, 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:26.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:26.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:26.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 71.331391792000005, 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:26.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:26.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:26.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:26.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:26.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:26.915 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:26.916 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:26.917 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:26.917 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:26.918 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:26.918 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:26.918 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:26.914+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:26.919 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:26.919 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:26.919 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:26.919 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:26.920 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:26.920 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:26.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:26.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:11:27.019 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:27.017+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:27.097 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:27.117 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:27.119 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:27.120 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:27.117+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:27.220 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:27.217+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:27.241 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:27.241 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:27.255 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:27.321 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:27.317+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:27.377 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:27.377 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:27.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:27.439 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:27.437+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:27.521 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:27.521 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:27.522 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:27.517+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:27.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:27.623 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:27.618+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:27.706 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:27.706 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:27.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:27.724 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:27.721+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:27.825 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:27.821+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:27.894 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:27.894 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:27.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:27.951 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:27.949+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:27.975 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:11:28.026 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:28.022+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:28.086 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:28.086 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:28.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:28.127 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:28.122+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:28.228 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:28.225+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:28.228 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:28.229 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:28.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:28.329 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:28.326+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:28.412 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:28.413 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:28.427 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:28.429 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:28.425+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:28.494 INFO:teuthology.orchestra.run.smithi005.stdout:get_pg_num_history pg_num_history pg_num_history(e182 pg_nums {1={13=1},2={16=8},3={21=32}} deleted_pools ) 2024-09-14T07:11:28.494 INFO:teuthology.orchestra.run.smithi005.stdout:Importing pgid 3.1f 2024-09-14T07:11:28.495 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f882f4e5:::rbd_data.116170297dfe.0000000000000009:head# 2024-09-14T07:11:28.495 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.501 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f8928d1a:::rbd_data.1161a9b4f57.0000000000000036:head# 2024-09-14T07:11:28.501 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 7=[]:{} 2024-09-14T07:11:28.501 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f8a9ac2c:::rbd_data.1161a6ea4953.00000000000000d1:1b# 2024-09-14T07:11:28.503 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f8a9ac2c:::rbd_data.1161a6ea4953.00000000000000d1:head# 2024-09-14T07:11:28.503 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 1b=[]:{1b=[1b]} 2024-09-14T07:11:28.503 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f9065bcc:::rbd_data.116150b87d44.0000000000000001:head# 2024-09-14T07:11:28.511 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.530 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:28.525+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:28.557 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:28.557 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:28.571 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:28.585 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f925b192:::rbd_data.1161ca0f0ebc.0000000000000050:head# 2024-09-14T07:11:28.585 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2d=[]:{} 2024-09-14T07:11:28.588 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f9b02d91:::rbd_data.1161605482a1.0000000000000004:head# 2024-09-14T07:11:28.591 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.631 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:28.629+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:28.647 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f9b060bf:::rbd_data.1161ca0f0ebc.0000000000000043:head# 2024-09-14T07:11:28.647 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2d=[]:{} 2024-09-14T07:11:28.650 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:f9bbddff:::rbd_data.1161ce824eb3.0000000000000043:head# 2024-09-14T07:11:28.650 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 24=[]:{} 2024-09-14T07:11:28.651 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fa01a741:::rbd_data.1161a0b955f2.0000000000000040:head# 2024-09-14T07:11:28.651 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.653 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:b# 2024-09-14T07:11:28.672 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-14T07:11:28.691 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:28.691 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:28.706 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:28.723 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head# 2024-09-14T07:11:28.725 INFO:teuthology.orchestra.run.smithi005.stdout:snapset b=[]:{b=[b]} 2024-09-14T07:11:28.755 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:28.754+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:28.761 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fa52177d:::rbd_data.11611fdb64aa.0000000000000020:head# 2024-09-14T07:11:28.761 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 1a=[]:{} 2024-09-14T07:11:28.763 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fa654f18:::rbd_id.image_client.0:head# 2024-09-14T07:11:28.764 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.764 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fa782143:::rbd_data.1161a6fd7139.0000000000000034:23# 2024-09-14T07:11:28.767 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fa782143:::rbd_data.1161a6fd7139.0000000000000034:head# 2024-09-14T07:11:28.767 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 23=[]:{23=[23]} 2024-09-14T07:11:28.768 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fb633a10:::rbd_data.1161a9b4f57.0000000000000081:head# 2024-09-14T07:11:28.768 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 7=[]:{} 2024-09-14T07:11:28.769 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fbac3711:::rbd_data.1161a9b4f57.000000000000005d:head# 2024-09-14T07:11:28.769 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 7=[]:{} 2024-09-14T07:11:28.769 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fbbfc85c:::rbd_data.1161a6fd7139.0000000000000007:head# 2024-09-14T07:11:28.769 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 23=[]:{} 2024-09-14T07:11:28.770 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fbef71f2:::rbd_data.116155d7d34e.0000000000000009:head# 2024-09-14T07:11:28.770 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 12=[]:{} 2024-09-14T07:11:28.772 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fc2fb453:::rbd_data.116148ee4e32.0000000000000031:head# 2024-09-14T07:11:28.772 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.777 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fcb62c1a:::rbd_data.116148ee4e32.0000000000000009:head# 2024-09-14T07:11:28.777 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.786 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fcd26671:::rbd_data.1161a9b4f57.0000000000000045:head# 2024-09-14T07:11:28.786 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 7=[]:{} 2024-09-14T07:11:28.787 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fd7add64:::rbd_data.1161dbabc1ae.00000000000000d4:head# 2024-09-14T07:11:28.787 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.789 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fdbc7aa9:::rbd_data.116167e24a2d.000000000000000f:head# 2024-09-14T07:11:28.789 INFO:teuthology.orchestra.run.smithi005.stdout:snapset a=[]:{} 2024-09-14T07:11:28.789 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fdc075a7:::rbd_data.1161ce824eb3.0000000000000040:head# 2024-09-14T07:11:28.790 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 24=[]:{} 2024-09-14T07:11:28.790 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fddcb068:::rbd_data.1161a9b4f57.0000000000000060:head# 2024-09-14T07:11:28.790 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 7=[]:{} 2024-09-14T07:11:28.791 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:fe62e861:::rbd_data.1161ec00b0f6.000000000000003c:head# 2024-09-14T07:11:28.791 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-09-14T07:11:28.794 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ff7206e7:::rbd_data.1161ca0f0ebc.0000000000000051:head# 2024-09-14T07:11:28.794 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2d=[]:{} 2024-09-14T07:11:28.795 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:10# 2024-09-14T07:11:28.827 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:28.827 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:28.833 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:28.830+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:28.842 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:28.902 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head# 2024-09-14T07:11:28.902 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 10=[]:{10=[10]} 2024-09-14T07:11:28.906 INFO:teuthology.orchestra.run.smithi005.stdout:write_pg epoch 181 info 3.1f( v 177'362 (0'0,177'362] local-lis/les=88/89 n=31 ec=21/21 lis/c=88/88 les/c/f=89/89/0 sis=88) 2024-09-14T07:11:28.908 INFO:teuthology.orchestra.run.smithi005.stdout:Import successful 2024-09-14T07:11:28.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:28.971 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:28.969+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:29.018 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:29.018 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:29.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:29.034 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:29.029+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:29.055 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:29.073 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:29.074 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:29.090 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:29.091 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:29.108 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:29.109 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:29.127 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:29.127 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:29.135 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:29.133+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:29.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:29.150 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:29.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:29.164 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:29.164 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:29.165 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:29.165 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:29.165 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:29.181 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:29.236 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:29.234+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 192.00237398600001, 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:29.302 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:29.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:29.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:29.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.308 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 179.14289839899999, 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:29.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:29.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 179.14279611399999, 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:29.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:29.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 178.44374427299999, 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.319 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.958732712, 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.324 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:29.328 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.95869864299999, 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:29.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:29.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:29.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.95865529700001, 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:29.332 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.95862177399999, 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.336 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:29.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.958574524, 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.340 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:29.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:29.344 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.95846027600001, 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.348 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.95835058099999, 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:29.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 142.95828443799999, 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.353 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:29.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 86.723208589999999, 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.357 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:29.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 77.436495153999999, 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:29.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.368 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.820480111999998, 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:29.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:29.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:29.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:29.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.820416413000004, 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:29.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:29.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.380 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.820373079000007, 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:29.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:29.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:29.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:29.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.820316199999994, 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.388 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.393 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.820249704999995, 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:29.394 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:29.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:29.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:29.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.817365367999997, 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:29.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.405 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:29.406 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:29.407 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:29.401+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:29.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /var/log/ceph/exp.3.1f.0 2024-09-14T07:11:29.507 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:29.502+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:29.575 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:29.575 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:29.575 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:29.575 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:29.589 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:29.632 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:29.630+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:29.709 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:29.705+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:29.749 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:29.749 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:29.749 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:29.749 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:29.749 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:29.763 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:29.810 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:29.806+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 217.05576262599999, 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:29.889 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:29.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:29.894 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 217.054988278, 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:29.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 188.04380527999999, 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.900 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:29.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 187.746877074, 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:29.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.911 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.51197325000001, 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:29.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.51141145899999, 2024-09-14T07:11:29.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:29.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:29.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:29.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.51091689200001, 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.922 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.510299473, 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.926 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:29.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.50987121399999, 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:29.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:29.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.50947535, 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:29.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:29.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.508883674, 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:29.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:29.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.505308931, 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:29.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.948 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.505269578, 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:29.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.952 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.50523179800001, 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:29.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.505195095, 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:29.957 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:29.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.915823161000006, 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:29.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:29.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:29.966 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.911150728999999, 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:29.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.911068236999995, 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.971 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.975 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.910907116000004, 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:29.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:29.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.910730208000004, 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:29.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:29.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:29.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:29.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:29.984 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:29.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:29.986 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:29.985+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:30.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:11:30.086 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:30.082+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:30.120 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:26434 3.16 e197/163 3:6f2c36e3:::rbd_header.1161b0eeaf54:head v 197'608, mlcod=197'606)", 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:30.119733+0000", 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00091641599999999998, 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00093861499999999996, 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119733+0000", 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119733+0000", 2024-09-14T07:11:30.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119734+0000", 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2869999999999999e-06 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119748+0000", 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3594000000000001e-05 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119766+0000", 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8828e-05 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119776+0000", 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9010000000000006e-06 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119836+0000", 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9571999999999998e-05 2024-09-14T07:11:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:30.119854+0000", 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8627e-05 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:11:30.125 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:30.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:30.187 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:30.186+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:30.257 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:30.257 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:30.257 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:30.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:30.319 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:30.317+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:30.389 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:30.385+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:30.393 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 188.712415964, 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:30.394 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:30.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321410329, 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.399 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.402 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32137805400001, 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:30.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32131995099999, 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:30.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:30.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321279049, 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:30.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321217241, 2024-09-14T07:11:30.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:30.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:30.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.419 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32118292499999, 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:30.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:30.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:30.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321164254, 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:30.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32114021800001, 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.434 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:30.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:30.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32112018399999, 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:30.438 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:30.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.442 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32110534899999, 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:30.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321090723, 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:30.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.450 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32107622199999, 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:30.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:30.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321058467, 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.455 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:30.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.458 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321043573, 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:30.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32102946200001, 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:30.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.321007533, 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.467 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 143.32098622199999, 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:30.471 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.475 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 139.03363532099999, 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:30.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:30.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:30.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.531966932000003, 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:30.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:30.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:30.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:30.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:30.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:30.487 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:30.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:30.489 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:30.486+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:30.590 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:30.585+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:30.664 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:30.665 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:30.665 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:30.680 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:30.691 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:30.686+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:30.792 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:30.789+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:30.850 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:30.850 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:30.850 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:30.865 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:30.892 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:30.890+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:30.994 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:30.990+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:31.004 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:31.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 193.738035924, 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:31.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.045 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 193.69304561800001, 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.051 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 189.39111316200001, 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:31.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:31.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:31.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 189.06978542799999, 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:31.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:31.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.063 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.632657029, 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:31.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.63264637099999, 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:31.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:31.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.63263135700001, 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:31.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.078 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.63262044199999, 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:31.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:31.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.083 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.63260893200001, 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.63259638900001, 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.62574979300001, 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.625702012, 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.625688545, 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.62567757299999, 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.625666811, 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.62565075699999, 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.62563418600001, 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.62562241000001, 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 144.625611733, 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.464435308000006, 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:31.159 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:31.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:31.161 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:31.158+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:31.162 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:31.261 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:31.258+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:31.305 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:31.306 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:31.320 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:31.341 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:31.360 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:31.360 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:31.361 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:31.357+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:31.377 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:31.395 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:31.413 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:31.431 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:31.435 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:31.435 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:31.462 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:31.458+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:31.494 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:31.494 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:31.508 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:31.596 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:31.594+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:31.665 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:31.661+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:31.672 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:31.686 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:31.772 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:31.770+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:31.854 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:31.854 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:31.866 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:31.862+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:31.868 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:31.968 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:31.965+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:32.046 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:32.047 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:32.063 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:32.107 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:32.106+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:32.169 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:32.166+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:32.190 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:32.191 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:32.205 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:32.296 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:32.294+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:32.370 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:32.365+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:32.373 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:32.373 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:32.387 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:32.472 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:32.469+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:32.555 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:32.556 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:32.569 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:32.572 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:32.569+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:32.673 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:32.670+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:32.687 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:32.687 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:32.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:32.774 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:32.774+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:32.836 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:32.837 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:32.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:32.874 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:32.870+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:32.975 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:32.974+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:32.979 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:32.980 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:32.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:33.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:11:33.076 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:33.074+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:33.130 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:33.130 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:33.147 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:33.195 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:33.193+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:33.277 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:33.277 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:33.278 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:33.273+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:33.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:33.379 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:33.374+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:33.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:33.462 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:33.462 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:33.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:33.523 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:33.521+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:33.570 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:33.580 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:33.577+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:33.589 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:33.606 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:33.644 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:33.644 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:33.675 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 196.37224069000001, 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 196.327250384, 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 192.02531792799999, 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 191.703990194, 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.26686179500001, 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.266851137, 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.26683612299999, 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.266825208, 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.26681369799999, 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.266801155, 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.25995455899999, 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.25990677799999, 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.25989331100001, 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.259882339, 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:33.744 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:33.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:33.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.25987157700001, 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.749 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.259855523, 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:33.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.259838952, 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:33.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:33.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.25982717599999, 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:33.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:33.766 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.25981649900001, 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:33.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.098640074000002, 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:33.771 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:33.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:33.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:33.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.777 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:33.778 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:33.778 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:33.779 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:33.779 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:33.780 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:33.780 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:33.781 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:33.781 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:33.782 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:33.782 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:33.777+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:33.881 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:33.878+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:33.913 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:33.913 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:33.913 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:33.913 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:33.928 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:34.020 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:34.018+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:34.035 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:11:34.082 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:34.078+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:34.101 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:34.101 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:34.101 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:34.101 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:34.101 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:34.116 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:34.212 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:34.210+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:34.285 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:34.282+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 241.992681675, 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:34.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:34.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.09868849099999, 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:34.303 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:34.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:34.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:34.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.09856861200001, 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:34.312 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:34.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:34.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:34.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:34.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.081343492, 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:34.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:34.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:34.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 143.408749105, 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.325 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:34.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:34.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63461105100001, 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:34.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:34.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63457735099999, 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.335 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.339 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63455670799999, 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:34.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63431149100001, 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.344 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:34.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:34.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63427977200001, 2024-09-14T07:11:34.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:34.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63425801299999, 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:34.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:34.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:34.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:34.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.634225934, 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.357 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:34.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:34.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.361 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63305472799999, 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.362 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:34.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.633031765, 2024-09-14T07:11:34.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:34.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63301047499999, 2024-09-14T07:11:34.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:34.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:34.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 141.63299025200001, 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.376 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:34.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 64.505264147999995, 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:34.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.506019035000001, 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:34.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:34.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.505971643000002, 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:34.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:34.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.395 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.505930116000002, 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:34.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.400 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.401 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.401 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.401 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:34.402 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:34.403 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:34.398+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:34.503 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:34.498+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:34.530 INFO:tasks.thrashosds.thrasher:in_osds: [3, 5, 2, 4, 0, 1] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-14T07:11:34.530 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-14T07:11:34.531 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T07:11:34.531 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T07:11:34.534 INFO:tasks.ceph.osd.0:Started 2024-09-14T07:11:34.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:34.569 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:34.569 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:34.570 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:34.570 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:34.584 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:34.628 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:34.626+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:config diff 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:config diff get 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:config get 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:config help [] 2024-09-14T07:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:config set ... 2024-09-14T07:11:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:admin_socket: invalid command 2024-09-14T07:11:34.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:11:34.662 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-14T07:11:34.704 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:34.702+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:34.748 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:34.748 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:34.749 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:34.763 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:34.804 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:34.802+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:34.906 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:34.902+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:34.929 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 197.63134984499999, 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:34.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:34.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:34.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 184.771874258, 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.935 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:34.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:34.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:34.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 184.771771973, 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:34.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:34.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 184.072720132, 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:34.944 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:34.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.58770857100001, 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.949 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:34.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:34.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.587674502, 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:34.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.58763115599999, 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.960 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.587597633, 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:34.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:34.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:34.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.58755038300001, 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:34.964 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.58743613499999, 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.968 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:34.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:34.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.58732644, 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:34.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:34.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.587260297, 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:34.976 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:34.979 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 92.352184449000006, 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:34.980 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.984 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 83.065471013000007, 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.985 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:34.991 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.449455971000006, 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:34.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:34.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:34.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:34.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:34.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.449392271999997, 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:34.998 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:34.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:35.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:35.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.003 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.449348938, 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:35.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:35.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:35.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.449292059000001, 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:35.009 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:35.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:35.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:35.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.449225564000002, 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.015 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:35.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:35.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:35.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 79.446341227000005, 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:35.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:35.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:35.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.026 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:35.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:35.028 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:35.026+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:35.128 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:35.126+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:35.209 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:35.209 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:35.209 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:35.209 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:35.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:35.268 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:35.266+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:35.329 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:35.325+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:35.391 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:35.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:35.391 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:35.391 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:35.391 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:35.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:35.452 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:35.450+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 222.69279226099999, 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:35.526 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:35.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:35.531 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 222.692017913, 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:35.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 193.68083491499999, 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:35.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:35.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:35.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 193.383906709, 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.543 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:35.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:35.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:35.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:35.548 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14900288499999, 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:35.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:35.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.552 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14844109399999, 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:35.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.557 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14794652699999, 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:35.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14732910800001, 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:35.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:35.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:35.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14690084899999, 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:35.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14650498500001, 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:35.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14591330900001, 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:35.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:35.580 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14233856600001, 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:35.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.142299213, 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:35.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14226143299999, 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:35.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:35.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 149.14222473000001, 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.594 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:35.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.598 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.552852795999996, 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:35.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:35.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.603 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.548180364000004, 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:35.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:35.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:35.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.548097872, 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:35.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.547936750999995, 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:35.612 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:35.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.547759842999994, 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:35.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.621 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.621 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.621 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:35.621 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:35.621 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:35.622 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:35.618+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:35.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:35.723 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:35.721+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:35.758 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:35.758 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:35.758 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:35.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:35.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:35.825 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:35.822+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:35.926 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:35.921+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:35.936 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:35.933+0000 7f9bc60b37c0 -1 Falling back to public interface 2024-09-14T07:11:35.946 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:35.946 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:35.946 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:35.946 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:35.946 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:35.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:35.971 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:35.971 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:35.988 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:35.989 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:36.009 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:36.029 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:36.025+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:36.031 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:36.056 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:36.062 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:36.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:36.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:11:36.094 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:36.094 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:36.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:36.134 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:36.130+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:36.231 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:36.226+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:36.276 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:36.276 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:36.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:36.335 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:36.333+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:36.408 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:36.408 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:36.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:36.463 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:36.461+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:36.534 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:36.529+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:36.591 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:36.592 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:36.610 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:36.656 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:36.654+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:36.737 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:36.733+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:36.742 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:36.742 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:36.756 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:36.838 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:36.833+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:36.922 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:36.922 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:36.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:36.939 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:36.937+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:37.040 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:37.038+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:37.049 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:11:37.059 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:37.059 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:37.076 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:37.141 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:37.137+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:37.242 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:37.238+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:37.251 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:37.251 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:37.266 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:37.342 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:37.337+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:37.429 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:37.430 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:37.443 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:37.438+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:37.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:37.543 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:37.541+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:37.611 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:37.611 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:37.625 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:37.672 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:37.670+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:37.743 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:37.743 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:37.744 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:37.742+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:37.758 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:37.847 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:37.846+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:37.923 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:37.924 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:37.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:37.947 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:37.941+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:38.048 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:38.046+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:38.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:38.105 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:38.105 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:38.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:38.164 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:38.162+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:38.204 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:38.225 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:38.243 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:38.250 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:38.246+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:38.254 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:38.254 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:38.255 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:38.268 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:38.268 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:38.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:38.291 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:38.311 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:38.311 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:38.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:38.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:38.352 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:38.350+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:38.395 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:38.395 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:38.395 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:38.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:38.452 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:38.450+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:38.532 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 196.85018658999999, 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9247 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45918095499999, 2024-09-14T07:11:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120347079, 2024-09-14T07:11:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9247 2024-09-14T07:11:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070791+0000", 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6499999999999999e-07 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070794+0000", 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070805+0000", 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1122e-05 2024-09-14T07:11:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070813+0000", 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3239999999999997e-06 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191106+0000", 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120293604 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191138+0000", 2024-09-14T07:11:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1486000000000003e-05 2024-09-14T07:11:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45914868, 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45909057700001, 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45904967499999, 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45898786699999, 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45895355100001, 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45893487999999, 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.458910844, 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45889081000001, 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45887597500001, 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45886134899999, 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45884684800001, 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45882909299999, 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45881419899999, 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.458800088, 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45877815899999, 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 151.45875684800001, 2024-09-14T07:11:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 147.17140594700001, 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:38.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:38.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 40.669737558000001, 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:38.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:38.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:38.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:38.630 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:38.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:38.632 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:38.629+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:38.732 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:38.730+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:38.800 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:38.800 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:38.801 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:38.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:38.859 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:38.857+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:38.931 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:38.931 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:38.932 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:38.934 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:38.929+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:38.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:39.035 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:39.030+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:39.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:11:39.130 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:39.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 201.828447012, 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:39.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:39.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:39.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 201.78345670600001, 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:39.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 197.48152425000001, 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:39.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:39.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.148 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 197.16019651600001, 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:39.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:39.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:39.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.723068117, 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.72305745899999, 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:39.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:39.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:39.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.723042445, 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:39.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.168 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.72303152999999, 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:39.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:39.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.72302002000001, 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:39.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:39.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.72300747700001, 2024-09-14T07:11:39.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.71616088100001, 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.182 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.7161131, 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.716099633, 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:39.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.71608866099999, 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:39.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.716077899, 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:39.199 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:39.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:39.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.71606184500001, 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:39.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:39.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.71604527400001, 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:39.208 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:39.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:39.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.716033498, 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:39.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:39.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.716022821, 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:39.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 96.554846396000002, 2024-09-14T07:11:39.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:39.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:39.228 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:39.228 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:39.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:39.230 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:39.226+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:39.330 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:39.326+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:39.361 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:39.361 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:39.361 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:39.361 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:39.375 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:39.464 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:39.462+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:39.532 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:39.530+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:39.541 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:39.541 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:39.541 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:39.541 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:39.541 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:39.555 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:39.644 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:39.642+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:39.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 247.41856860600001, 2024-09-14T07:11:39.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:39.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 171.524575422, 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:39.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:39.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:39.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 171.52445554299999, 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:39.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.745 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 171.50723042300001, 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.750 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 148.83463603600001, 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:39.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:39.756 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.06049798199999, 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.760 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.060464282, 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.761 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:39.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:39.765 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.060443639, 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:39.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:39.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.06019842200001, 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:39.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:39.774 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.06016670299999, 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:39.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.060144944, 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.783 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.06011286500001, 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:39.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:39.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.058941659, 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:39.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.792 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.05891869600001, 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.796 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.058897406, 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.797 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:39.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 147.05887718299999, 2024-09-14T07:11:39.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:39.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 69.931151079000003, 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:39.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:39.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:39.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.931905966000002, 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:39.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:39.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.816 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.931858574000003, 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.821 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.931817047000003, 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:39.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:39.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.826 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:39.827 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:39.827 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:39.828 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:11:39.829 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-09-14T07:11:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-09-14T07:11:39.829 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator fragmentation block 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator fragmentation histogram block [] [] 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator score block 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:config diff 2024-09-14T07:11:39.830 INFO:teuthology.orchestra.run.smithi005.stderr:admin_socket: invalid command 2024-09-14T07:11:39.830 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-14T07:11:39.830 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:39.825+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:39.929 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:39.925+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:40.000 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:40.001 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:40.001 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:40.001 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:40.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:40.064 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:40.062+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:11:40.129 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:40.125+0000 7f9bc60b37c0 -1 osd.0 182 log_to_monitors true 2024-09-14T07:11:40.130 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:40.126+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:40.146 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:40.146 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:40.147 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:40.147 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:40.161 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:40.252 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:40.250+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:40.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:40.331 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:40.330+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:40.341 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:40.341 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:40.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 203.04209759899999, 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:40.342 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:40.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:40.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:40.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:40.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:40.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:40.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 190.182622012, 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:40.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:40.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 190.182519727, 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.355 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:40.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 189.483467886, 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.360 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:40.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.364 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.99845632500001, 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:40.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:40.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.998422256, 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:40.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.99837891000001, 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:40.373 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:40.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.377 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.998345387, 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:40.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.381 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.99829813700001, 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:40.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.99818388899999, 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.386 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:40.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.998074194, 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.390 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:40.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:40.394 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 153.998008051, 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:40.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:40.398 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.762932203000005, 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:40.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 88.476218767000006, 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.404 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:40.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:40.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:40.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.410 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.860203725000005, 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:40.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:40.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:40.416 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.860140025999996, 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:40.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:40.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:40.422 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.860096691999999, 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:40.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:40.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:40.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.860039813, 2024-09-14T07:11:40.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:40.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:40.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:40.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:40.434 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.859973318000002, 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:40.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:40.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:40.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:40.440 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 84.857088981000004, 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:40.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:40.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:40.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:40.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:40.447 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:40.448 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:40.449 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:40.446+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-14T07:11:40.474 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:40.475 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:40.492 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:40.512 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:40.533 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:40.549 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:40.545+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:40.553 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:40.553 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:40.557 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-14T07:11:40.558 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-14T07:11:40.623 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:40.624 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:40.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:40.684 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:40.682+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:40.750 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:40.746+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:40.762 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:40.762 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:40.776 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:40.852 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:40.849+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:40.945 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:40.945 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:40.952 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:40.950+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:40.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:41.053 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:41.049+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:41.101 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:41.102 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:41.108 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:41.105+0000 7f9bb9e8e640 -1 osd.0 182 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T07:11:41.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:41.159 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:41.157+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:41.255 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:41.253+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:41.290 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:41.290 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:41.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:41.356 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:41.354+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:41.457 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:41.454+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:41.477 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:41.477 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:41.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:41.558 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:41.553+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:41.611 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:41.612 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:41.626 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:41.667 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:41.665+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:41.761 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:41.757+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:41.793 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:41.794 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:41.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:41.895 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:41.893+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:41.962 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:41.957+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:41.977 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:41.977 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:41.992 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:42.063 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:42.061+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:42.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:11:42.136 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:42.136 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:42.152 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:42.196 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:42.193+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:42.265 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:42.261+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:42.268 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:42.268 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:42.282 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:42.367 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:42.361+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:42.448 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:42.448 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:42.462 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:42.511 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:42.509+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:42.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:42.578 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:42.578 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:42.593 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:42.607 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:42.605+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:42.684 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:42.681+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:42.710 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:42.710 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-14T07:11:42.727 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:42.727 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:42.743 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:42.744 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:42.756 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:42.757 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:42.764 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:42.770 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:42.765+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:42.771 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:42.780 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:11:42.787 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:42.817 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:42.817 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:42.870 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:42.865+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:42.962 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:42.962 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:42.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:42.962 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:42.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 205.664030295, 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:42.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:42.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:42.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:42.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 192.80455470800001, 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:42.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:42.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:42.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:42.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:42.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 192.80445242299999, 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:42.974 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:42.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:42.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:42.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:42.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:42.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:42.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 192.10540058199999, 2024-09-14T07:11:42.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:42.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:42.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.62038902099999, 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:42.986 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:42.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:42.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.62035495200001, 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.991 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:42.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:42.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.620311606, 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:42.995 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:42.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.62027808299999, 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:42.999 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:43.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.003 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.62023083299999, 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.004 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.62011658500001, 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.008 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.009 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:43.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:43.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:43.012 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.62000689000001, 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:43.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.016 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 156.61994074699999, 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.017 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:43.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.38486489899999, 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.021 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:43.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 91.098151462999994, 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.026 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.027 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.028 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:43.029 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:43.030 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.031 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.032 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.482136421000007, 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:43.033 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.034 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.035 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.036 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:43.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.482072721999998, 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:43.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.040 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.041 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.042 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:43.043 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.482029388000001, 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.045 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:43.046 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.047 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.048 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:43.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.050 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.481972509000002, 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:43.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.052 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.053 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:43.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:43.056 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.481906014000003, 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:43.057 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:43.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:43.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 87.479021677000006, 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.063 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:43.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.069 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.069 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.069 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:43.070 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:43.071 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:43.065+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:43.171 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:43.169+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:43.243 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:43.243 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:43.243 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:43.243 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:43.258 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:43.272 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:43.269+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:43.374 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:43.373+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:43.422 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:43.422 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:43.422 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:43.436 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:43.475 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:43.473+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:43.554 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:43.554 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:43.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:43.554 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 230.72282607299999, 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:43.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:43.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 230.722051725, 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:43.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:43.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 201.71086872699999, 2024-09-14T07:11:43.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:43.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:43.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 201.413940521, 2024-09-14T07:11:43.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:43.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:43.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17903669699999, 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:43.577 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:43.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17847490599999, 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:43.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:43.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17798033899999, 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:43.586 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17736292000001, 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:43.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17693466099999, 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17653879700001, 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:43.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.604 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17594712100001, 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.608 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17237237800001, 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.609 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:43.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.613 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.172333025, 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:43.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17229524499999, 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:43.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:43.622 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.17225854200001, 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:43.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.626 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 95.582886607999995, 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:43.627 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 95.578214176000003, 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.631 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:43.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 95.578131683999999, 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:43.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:43.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:43.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 95.577970562999994, 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:43.640 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:43.644 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 95.577793654999994, 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:43.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:43.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:43.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:43.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:43.649 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:43.651 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:43.649+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:43.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:43.752 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:43.749+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:43.789 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:43.789 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28319 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e210)", 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96566575300000002, 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.96568389700000001, 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for peered", 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:43.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28319 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:43.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823897+0000", 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4600000000000004e-07 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823900+0000", 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4910000000000001e-06 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823908+0000", 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5229999999999998e-06 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823912+0000", 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.143e-06 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823975+0000", 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2892999999999996e-05 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823987+0000", 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2215000000000001e-05 2024-09-14T07:11:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:11:43.794 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:43.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:43.854 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:43.849+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:43.954 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:43.949+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:43.979 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:43.979 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:43.979 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:43.979 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:43.979 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:43.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:44.055 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:44.049+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:44.156 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:44.153+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:44.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 202.48829832300001, 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:44.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09726041299999, 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:44.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09720231, 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:44.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:44.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09716140800001, 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:44.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:44.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09709960000001, 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:44.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:44.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.097065284, 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.194 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.097046613, 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09702257699999, 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.097002543, 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:44.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:44.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.096987708, 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09697308200001, 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:44.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:44.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.096958581, 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:44.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09694082600001, 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.221 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:44.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.096925932, 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:44.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09691182099999, 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.229 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:44.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.09688989200001, 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.233 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:44.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 157.096868581, 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:44.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 152.80951768, 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:44.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.307849290999997, 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:44.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:44.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:44.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28319 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e210)", 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.3441872589999999, 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.976578579, 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28319 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.253 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823897+0000", 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4600000000000004e-07 2024-09-14T07:11:44.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823900+0000", 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4910000000000001e-06 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823908+0000", 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5229999999999998e-06 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823912+0000", 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.143e-06 2024-09-14T07:11:44.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823975+0000", 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2892999999999996e-05 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823987+0000", 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2215000000000001e-05 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.800096+0000", 2024-09-14T07:11:44.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.97610919500000004 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.800475+0000", 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00037847299999999999 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.257 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:44.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:44.259 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:44.253+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:44.359 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:44.357+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:44.433 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:44.433 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:44.434 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:44.434 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:44.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:44.491 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:44.489+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:44.561 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:44.557+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:44.620 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:44.621 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:44.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:44.662 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:44.661+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:44.763 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:44.761+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:44.816 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 207.510586362, 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:44.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:44.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:44.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:44.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 207.46559605600001, 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:44.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:44.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:44.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 203.16366360000001, 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.830 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:44.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:44.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:44.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 202.84233586600001, 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:44.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:44.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.405207467, 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:44.842 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:44.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.40519680899999, 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.847 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:44.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:44.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.405181795, 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.855 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.40517088000001, 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.859 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.40515937000001, 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.864 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.40514682700001, 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:44.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:44.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.39830023100001, 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:44.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.873 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.39825245, 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.398238983, 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:44.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.39822801099999, 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.883 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:44.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.398217249, 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.891 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.39820119500001, 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:44.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:44.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.39818462400001, 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.398172848, 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:44.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:44.904 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 158.398162171, 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:44.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.909 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 102.236985746, 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:44.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:44.916 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:44.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:11:44.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:44.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:44.919 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:44.920 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:44.917+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:45.021 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:45.017+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:45.050 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:45.050 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:45.050 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:45.050 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:45.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:45.102 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28504 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e212)", 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:44.862342+0000", 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.24064248699999999, 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.24067486599999999, 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for peered", 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28504 2024-09-14T07:11:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862342+0000", 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862342+0000", 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862345+0000", 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.14e-06 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862361+0000", 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5889999999999999e-05 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862398+0000", 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6582000000000003e-05 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862420+0000", 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2498999999999999e-05 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862570+0000", 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015011399999999999 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.862588+0000", 2024-09-14T07:11:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7353000000000001e-05 2024-09-14T07:11:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-14T07:11:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-09-14T07:11:45.107 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:45.114 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:45.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T07:11:45.133 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:45.150 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:45.152 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:45.149+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:45.168 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:45.189 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:45.189 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:45.208 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:45.222 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:45.217+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:45.232 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:45.232 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:45.247 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:45.332 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:45.329+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:45.406 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:45.406 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:45.421 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:45.464 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:45.461+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:45.526 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:45.521+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:45.583 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:45.583 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:45.598 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:45.626 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:45.621+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:45.728 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:45.725+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:45.763 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:45.763 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:45.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-09-14T07:11:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T07:11:45.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T07:11:45.828 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:45.825+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:45.908 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:45.908 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:45.923 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:45.929 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:45.925+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:46.031 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:46.025+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:46.094 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:46.094 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:46.113 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:46.156 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:46.153+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:46.233 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:46.229+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:46.284 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:46.284 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:46.298 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:46.344 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:46.341+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:46.421 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:46.421 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:46.435 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:46.433+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:46.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:46.536 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:46.533+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:46.566 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:46.566 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:46.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:46.637 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:46.633+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:46.709 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:46.709 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:46.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:46.738 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:46.733+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T07:11:46.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T07:11:46.839 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:46.837+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:46.912 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:46.912 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:46.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:46.940 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:46.937+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:47.041 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:47.037+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:47.090 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:47.090 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:47.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:47.141 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:47.137+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:47.219 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:47.219 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:47.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:47.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:47.242 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:47.237+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:47.343 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:47.341+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:47.371 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:47.390 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:47.397 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 205.71688597599999, 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9248 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32584806599999, 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120386882, 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9248 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070823+0000", 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070824+0000", 2024-09-14T07:11:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000001e-07 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070826+0000", 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070836+0000", 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3270000000000007e-06 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070842+0000", 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7009999999999999e-06 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191196+0000", 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12035347 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191210+0000", 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3967e-05 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9250 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32578996300001, 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120388917, 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9250 2024-09-14T07:11:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070881+0000", 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070882+0000", 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0160000000000001e-06 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070886+0000", 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5640000000000001e-06 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070900+0000", 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4714e-05 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070908+0000", 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1259999999999998e-06 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191254+0000", 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120345177 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191270+0000", 2024-09-14T07:11:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.632e-05 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32574906100001, 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32568725300001, 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.325652937, 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32563426600001, 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32561023, 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32559019600001, 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32557536100001, 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32556073500001, 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.325546234, 2024-09-14T07:11:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32552847900001, 2024-09-14T07:11:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32551358500001, 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.325499474, 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.32547754500001, 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 160.325456234, 2024-09-14T07:11:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 156.038105333, 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 49.536436944000002, 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28319 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e210)", 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.5727749119999999, 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.976578579, 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28319 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823897+0000", 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4600000000000004e-07 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823900+0000", 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4910000000000001e-06 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823908+0000", 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5229999999999998e-06 2024-09-14T07:11:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823912+0000", 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.143e-06 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823975+0000", 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2892999999999996e-05 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823987+0000", 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2215000000000001e-05 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.800096+0000", 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.97610919500000004 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.800475+0000", 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00037847299999999999 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:47.484 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:47.485 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:47.485 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:47.486 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:47.487 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:47.487 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:47.487 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:47.488 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:47.488 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:47.488 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:47.489 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:47.481+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:47.587 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:47.585+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:47.657 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:47.657 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:47.657 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:47.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:47.715 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:47.713+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T07:11:47.789 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:47.785+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:47.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T07:11:47.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:11:47.854 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:47.854 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:47.854 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:47.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:47.915 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:47.913+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:47.990 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:47.985+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:48.034 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1911 3.8 3:171e9aa6:::rbd_data.11615839fdc9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14679040~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 210.73265581300001, 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.112914312, 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1911 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300307+0000", 2024-09-14T07:11:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.300309+0000", 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.632e-06 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345190+0000", 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044880903999999999 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345222+0000", 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1764999999999998e-05 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345236+0000", 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4452e-05 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345338+0000", 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101996 2024-09-14T07:11:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345534+0000", 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019598200000000001 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.347450+0000", 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019156710000000001 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.412786+0000", 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.065335624999999994 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413130+0000", 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034472099999999998 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413170+0000", 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9703000000000003e-05 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.413221+0000", 2024-09-14T07:11:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0861e-05 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4803.0:1913 3.1e 3:7aab7fb2:::rbd_data.11615839fdc9.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 14614528~41984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 210.68766550699999, 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.085295706999999998, 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4803", 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345297+0000", 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.345299+0000", 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.598e-06 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369018+0000", 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023718957999999998 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369077+0000", 2024-09-14T07:11:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9517e-05 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369121+0000", 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3467000000000002e-05 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369239+0000", 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117959 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.369448+0000", 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020964099999999999 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.371327+0000", 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001879014 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430079+0000", 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058751774999999999 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430515+0000", 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043590300000000002 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430554+0000", 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8829999999999999e-05 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:17.430593+0000", 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9045000000000003e-05 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:374 3.8 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15561216~52736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 206.38573305099999, 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.16776893600000001, 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 374 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647229+0000", 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.647232+0000", 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670213+0000", 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022980461000000001 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670267+0000", 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4004e-05 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670289+0000", 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1838999999999999e-05 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670484+0000", 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019511999999999999 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.670918+0000", 2024-09-14T07:11:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043416499999999997 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.677697+0000", 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067794800000000001 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.804944+0000", 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12724658 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814815+0000", 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0098708419999999995 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814914+0000", 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9382000000000005e-05 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814998+0000", 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4185999999999997e-05 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4703.0:675 3.18 3:1bc3a543:::rbd_data.1161ff1af5c4.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 4713472~7680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 206.06440531699999, 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.14214747999999999, 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4703", 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 675 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968557+0000", 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.968577+0000", 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0098999999999998e-05 2024-09-14T07:11:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983440+0000", 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014862428 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983491+0000", 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1634000000000001e-05 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983515+0000", 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3629e-05 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.983714+0000", 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019913999999999999 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.984166+0000", 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00045230100000000001 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.991113+0000", 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0069466040000000003 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.102265+0000", 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.111151709 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110496+0000", 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082313290000000008 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110618+0000", 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012153400000000001 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:22.110705+0000", 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7072999999999998e-05 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2303 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 430080~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62727691800001, 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071994250999999995, 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2303 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405686+0000", 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405687+0000", 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405690+0000", 2024-09-14T07:11:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405693+0000", 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477431+0000", 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071738337999999999 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477538+0000", 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010683099999999999 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477680+0000", 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000141848 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2304 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 452608~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62726626, 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072417851000000005, 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2304 2024-09-14T07:11:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405696+0000", 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405697+0000", 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.035e-06 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405701+0000", 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405707+0000", 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.905e-06 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477765+0000", 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072057255000000001 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.477920+0000", 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015488700000000001 2024-09-14T07:11:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478114+0000", 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019446700000000001 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2305 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 475136~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62725124599999, 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072689777999999997, 2024-09-14T07:11:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2305 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405711+0000", 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405712+0000", 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405713+0000", 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-09-14T07:11:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405716+0000", 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405719+0000", 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478223+0000", 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072504183999999999 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478299+0000", 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6172000000000002e-05 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478401+0000", 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000101914 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2306 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 497664~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.627240331, 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073091297, 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2306 2024-09-14T07:11:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405722+0000", 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405723+0000", 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.048e-06 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405727+0000", 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9670000000000003e-06 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405730+0000", 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478467+0000", 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072736865999999997 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478627+0000", 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000160209 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478813+0000", 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018611699999999999 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2307 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 520192~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.076 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62722882099999, 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073476626000000003, 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2307 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405734+0000", 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405735+0000", 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405740+0000", 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5349999999999998e-06 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405742+0000", 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7379999999999999e-06 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.478873+0000", 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073130334000000005 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479024+0000", 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150903 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479210+0000", 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000186715 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.080 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2308 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 542720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62721627799999, 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073771528000000003, 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2308 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.405746+0000", 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412318+0000", 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065722369999999999 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412331+0000", 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452e-05 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412349+0000", 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7872000000000001e-05 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412354+0000", 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479306+0000", 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.066951965000000002 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479396+0000", 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9932999999999994e-05 2024-09-14T07:11:48.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.479518+0000", 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121548 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2322 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 858112~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62036968199999, 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072135772000000001, 2024-09-14T07:11:48.085 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2322 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412593+0000", 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6399999999999998e-07 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412594+0000", 2024-09-14T07:11:48.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412597+0000", 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1729999999999999e-06 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412604+0000", 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6340000000000003e-06 2024-09-14T07:11:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484527+0000", 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.071922485999999994 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484618+0000", 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1380999999999998e-05 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484729+0000", 2024-09-14T07:11:48.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000110692 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2323 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 880640~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62032190100001, 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072498347000000005, 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2323 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412641+0000", 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412643+0000", 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5120000000000001e-06 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412647+0000", 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.25e-06 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412650+0000", 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9749999999999999e-06 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484813+0000", 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072162911999999996 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.484969+0000", 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000156303 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485139+0000", 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016986700000000001 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2324 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 903168~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.094 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62030843400001, 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072760478000000003, 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2324 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412654+0000", 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412656+0000", 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.305e-06 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412659+0000", 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412662+0000", 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7499999999999999e-06 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485249+0000", 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072587457999999994 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485318+0000", 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8621000000000006e-05 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485415+0000", 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6861999999999995e-05 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2325 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 925696~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.620297462, 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073024094999999997, 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2325 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412665+0000", 2024-09-14T07:11:48.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412666+0000", 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.156e-06 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412670+0000", 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412672+0000", 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7439999999999999e-06 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485482+0000", 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.072809199000000005 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485570+0000", 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8626000000000002e-05 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485689+0000", 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118949 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2326 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 948224~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62028670000001, 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073228915000000006, 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2326 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412676+0000", 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412681+0000", 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412685+0000", 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412688+0000", 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7420000000000002e-06 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485738+0000", 2024-09-14T07:11:48.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073050673999999996 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485813+0000", 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4580999999999996e-05 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485905+0000", 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1775999999999997e-05 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.107 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2327 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 970752~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62027064599999, 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073392983999999994, 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2327 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412692+0000", 2024-09-14T07:11:48.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412693+0000", 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412697+0000", 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.253e-06 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412699+0000", 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.683e-06 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.485954+0000", 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073254508999999995 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486007+0000", 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3488e-05 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486085+0000", 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7497999999999996e-05 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2328 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 993280~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62025407499999, 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073581482000000004, 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2328 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412708+0000", 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412709+0000", 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412710+0000", 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.294e-06 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412714+0000", 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5549999999999998e-06 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412716+0000", 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8109999999999999e-06 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486136+0000", 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073419784000000002 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486202+0000", 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5832000000000003e-05 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486290+0000", 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.797e-05 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2329 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1015808~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.116 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62024229900001, 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073743937999999995, 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2329 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412720+0000", 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412722+0000", 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1960000000000001e-06 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412725+0000", 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412727+0000", 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.605e-06 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486334+0000", 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073606313000000007 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486393+0000", 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8968999999999997e-05 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486464+0000", 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1449e-05 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:2330 3.7 3:e67632f3:::rbd_data.116150b87d44.0000000000000007:14 [read 1038336~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 161.62023162200001, 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073913237000000007, 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:48.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2330 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412731+0000", 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.35e-07 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412736+0000", 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000003e-06 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412740+0000", 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.642e-06 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.412742+0000", 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7659999999999999e-06 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486501+0000", 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.073759035000000001 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486565+0000", 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3510999999999996e-05 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:06.486644+0000", 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9128000000000005e-05 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5713.0:1137 3.7 3:e58ffc59:::rbd_object_map.11618d2dcb2b:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 105.459055197, 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.21056654699999999, 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5713", 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1137 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573907+0000", 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573911+0000", 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573927+0000", 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6126000000000001e-05 2024-09-14T07:11:48.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573974+0000", 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7178999999999997e-05 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.573998+0000", 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4070000000000002e-05 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575943+0000", 2024-09-14T07:11:48.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019449560000000001 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.575964+0000", 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1404e-05 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781018+0000", 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.20505410299999999 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.781340+0000", 2024-09-14T07:11:48.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032183500000000002 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.782040+0000", 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069950600000000002 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784215+0000", 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021753290000000002 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784299+0000", 2024-09-14T07:11:48.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4098999999999996e-05 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784411+0000", 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112011 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:02.784474+0000", 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2692000000000002e-05 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:48.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:48.132 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:48.133 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:48.134 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:48.129+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:48.234 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:48.229+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:48.265 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:48.265 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:48.265 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:48.279 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:48.336 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:48.333+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:48.438 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:48.433+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:48.441 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:48.441 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:48.441 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:48.442 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:48.442 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:48.456 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:48.538 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:48.533+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:48.584 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:48.584 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:48.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:48.584 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:48.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4177.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 256.280957204, 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.30569515400000002, 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4177", 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/3681722906", 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1 2024-09-14T07:11:48.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300504+0000", 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300507+0000", 2024-09-14T07:11:48.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829999999999999e-06 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300523+0000", 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5167000000000001e-05 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300546+0000", 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3697e-05 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300566+0000", 2024-09-14T07:11:48.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9633e-05 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300657+0000", 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999998e-05 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.300666+0000", 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.602921+0000", 2024-09-14T07:11:48.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.302254207 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603346+0000", 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042526700000000001 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.603804+0000", 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045820000000000002 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:48.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605722+0000", 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019179290000000001 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.605917+0000", 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000195088 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606079+0000", 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016235199999999999 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:32.606199+0000", 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000119672 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1375 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 180.38696401999999, 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0381789320000001, 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1375 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194497+0000", 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194498+0000", 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2860000000000001e-06 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194505+0000", 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5679999999999996e-06 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194521+0000", 2024-09-14T07:11:48.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5869999999999999e-05 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194531+0000", 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9920000000000002e-06 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194599+0000", 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8071000000000006e-05 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194605+0000", 2024-09-14T07:11:48.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0750000000000003e-06 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197412+0000", 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028074929999999999 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.197424+0000", 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.17e-05 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.218952+0000", 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.021527517 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222457+0000", 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0035054470000000001 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229744+0000", 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0072871180000000004 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229967+0000", 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022256000000000001 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.230272+0000", 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000304784 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231587+0000", 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001315477 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232437+0000", 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084965799999999999 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232516+0000", 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9708000000000003e-05 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.232676+0000", 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015960799999999999 2024-09-14T07:11:48.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1377 3.15 3:ad0cdfc7:::rbd_data.11613d73da60.0000000000000003:head [stat,truncate 5505024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 180.38684414100001, 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.037306603, 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:48.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1377 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194617+0000", 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194618+0000", 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1600000000000001e-07 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194621+0000", 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194630+0000", 2024-09-14T07:11:48.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5100000000000004e-06 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194639+0000", 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2840000000000006e-06 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194733+0000", 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3958999999999999e-05 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.194746+0000", 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3434999999999999e-05 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196850+0000", 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021036610000000002 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.196861+0000", 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1216000000000001e-05 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.215079+0000", 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0182184839999999 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.220756+0000", 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0056764390000000001 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228776+0000", 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0080202450000000005 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.228948+0000", 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172067 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.229283+0000", 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0003347 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231024+0000", 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017408079999999999 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231540+0000", 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051670899999999996 2024-09-14T07:11:48.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231705+0000", 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016454999999999999 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.231924+0000", 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021866999999999999 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5079.0:1384 3.1f 3:ffe6dfcb:::rbd_data.11613d73da60.0000000000000001:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 180.36961902100001, 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.011313031, 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5079", 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/413095907", 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1384 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211842+0000", 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211844+0000", 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4750000000000001e-06 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211849+0000", 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4360000000000001e-06 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211863+0000", 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4541e-05 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211872+0000", 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211935+0000", 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2868999999999997e-05 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:48.211945+0000", 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0633e-05 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222562+0000", 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.010617254 2024-09-14T07:11:48.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.222861+0000", 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029855099999999998 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:49.223155+0000", 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029408100000000002 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:8927 3.12 3:48fd69b1:::rbd_header.11613007faf8:head [watch ping cookie 140573341300448 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 157.697024634, 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.471718678, 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8927 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.612 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884436+0000", 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884438+0000", 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6670000000000001e-06 2024-09-14T07:11:48.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884446+0000", 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1629999999999996e-06 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884462+0000", 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5832999999999999e-05 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884481+0000", 2024-09-14T07:11:48.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8703e-05 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884580+0000", 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8900999999999994e-05 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:10.884590+0000", 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.9310000000000003e-06 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.352991+0000", 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.46840163800000001 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for degraded object", 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.353014+0000", 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2776999999999999e-05 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.355940+0000", 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029257630000000001 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356001+0000", 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1464000000000006e-05 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:11.356155+0000", 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000153838 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2682 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1175552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.92288658000001, 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105702384, 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2682 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658574+0000", 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658575+0000", 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658584+0000", 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4900000000000006e-06 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658592+0000", 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658600+0000", 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4579999999999993e-06 2024-09-14T07:11:48.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.763959+0000", 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105358259 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764069+0000", 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110661 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764277+0000", 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020745099999999999 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2684 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1064960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.92285287999999, 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106046311, 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.622 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2684 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658608+0000", 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658609+0000", 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1399999999999997e-07 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658611+0000", 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658617+0000", 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3219999999999998e-06 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658622+0000", 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.454e-06 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764323+0000", 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.105701301 2024-09-14T07:11:48.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764435+0000", 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000112422 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764654+0000", 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021914800000000001 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2686 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1087488~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.626 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.92283223699999, 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106263227, 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2686 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658629+0000", 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658632+0000", 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1670000000000002e-06 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658637+0000", 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7790000000000001e-06 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658644+0000", 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5719999999999999e-06 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764698+0000", 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10605429399999999 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764775+0000", 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6812999999999998e-05 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764892+0000", 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011707800000000001 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.630 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2688 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1110016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.92258702000001, 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106333131, 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2688 2024-09-14T07:11:48.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658874+0000", 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658875+0000", 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3500000000000005e-07 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658879+0000", 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658888+0000", 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2979999999999997e-06 2024-09-14T07:11:48.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658895+0000", 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9009999999999997e-06 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.764913+0000", 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10601872499999999 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765017+0000", 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103164 2024-09-14T07:11:48.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765207+0000", 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000190559 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2690 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1132544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.92255530099999, 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106669263, 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.635 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2690 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658906+0000", 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.13e-07 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658909+0000", 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.244e-06 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658916+0000", 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0090000000000001e-06 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658920+0000", 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7500000000000003e-06 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765257+0000", 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10633613 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765358+0000", 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101656 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765575+0000", 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000216861 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2692 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1155072~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:48.639 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.922533542, 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106922343, 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2692 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658928+0000", 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658930+0000", 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2639999999999998e-06 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658937+0000", 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0840000000000003e-06 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658950+0000", 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3138e-05 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765633+0000", 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106682736 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765734+0000", 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010069999999999999 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765850+0000", 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115969 2024-09-14T07:11:48.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2693 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1177600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.922501463, 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10710103999999999, 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.644 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2693 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.658960+0000", 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660088+0000", 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011284870000000001 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660100+0000", 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2041e-05 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660110+0000", 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6369999999999994e-06 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660117+0000", 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765885+0000", 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10576809400000001 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.765952+0000", 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7092000000000001e-05 2024-09-14T07:11:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766061+0000", 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000108367 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2695 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1067008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.92133025699999, 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465699999999, 2024-09-14T07:11:48.648 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2695 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660131+0000", 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660132+0000", 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3099999999999997e-07 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660134+0000", 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7089999999999998e-06 2024-09-14T07:11:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660141+0000", 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7390000000000002e-06 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660146+0000", 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.011e-06 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766093+0000", 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10594745699999999 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766151+0000", 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7531999999999997e-05 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766265+0000", 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000114478 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.652 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2696 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1089536~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.921307294, 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10631693, 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2696 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660154+0000", 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660156+0000", 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.114e-06 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660163+0000", 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5339999999999996e-06 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660168+0000", 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8500000000000002e-06 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766302+0000", 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10613465 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766367+0000", 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4388999999999996e-05 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766471+0000", 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010389 2024-09-14T07:11:48.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2698 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1112064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.921286004, 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10648363499999999, 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.657 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2698 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660175+0000", 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660176+0000", 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4200000000000001e-07 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660178+0000", 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0150000000000002e-06 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660184+0000", 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3929999999999996e-06 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660189+0000", 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7849999999999999e-06 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766502+0000", 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.106313666 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766560+0000", 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7410000000000001e-05 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766659+0000", 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8924000000000004e-05 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5125.0:2700 3.1b 3:df8d41b2:::rbd_data.116150b87d44.0000000000000004:14 [read 1134592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:48.661 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 155.92126578099999, 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10666848399999999, 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5125", 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2700 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:48.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.660195+0000", 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664605+0000", 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044101540000000003 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664611+0000", 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.609e-06 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664623+0000", 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2451e-05 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.664632+0000", 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.072e-06 2024-09-14T07:11:48.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766692+0000", 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.10206048500000001 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766757+0000", 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5258000000000004e-05 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:12.766864+0000", 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000106455 2024-09-14T07:11:48.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6007.0:836 3.15 3:a9ceeb39:::rbd_data.116144317fe8.000000000000000d:28 [sparse-read 290816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e143)", 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.793539676999998, 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2952674209999999, 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6007", 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/394559423", 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 836 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787921+0000", 2024-09-14T07:11:48.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787924+0000", 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6749999999999998e-06 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787932+0000", 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9529999999999999e-06 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787957+0000", 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4799000000000001e-05 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.787969+0000", 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.181e-05 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788047+0000", 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8837000000000004e-05 2024-09-14T07:11:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:29.788055+0000", 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.221e-06 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049813+0000", 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.26175819299999997 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:30.049822+0000", 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4199999999999996e-06 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082584+0000", 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0327621979999999 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.082866+0000", 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028194299999999999 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:31.083189+0000", 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032237200000000002 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:362 3.19 3:98df8df8:::rbd_data.116130782be1.0000000000000023:head [sparse-read 397312~126976] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.794294563999998, 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1019330439999999, 2024-09-14T07:11:48.672 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 362 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787166+0000", 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787168+0000", 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787175+0000", 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:11:48.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787187+0000", 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2500999999999999e-05 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787196+0000", 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0969999999999995e-06 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787289+0000", 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2780000000000003e-05 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787300+0000", 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-05 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.888845+0000", 2024-09-14T07:11:48.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.10154478 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889100+0000", 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00025441999999999999 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6259.0:367 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140415969864800 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.794247171999999, 2024-09-14T07:11:48.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.107012804, 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6259", 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2834127372", 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 367 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787214+0000", 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787215+0000", 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2099999999999996e-07 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787218+0000", 2024-09-14T07:11:48.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8199999999999998e-06 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787229+0000", 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0767e-05 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787237+0000", 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6429999999999995e-06 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787382+0000", 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014553400000000001 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787404+0000", 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.194e-05 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889680+0000", 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1022759419999999 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.889907+0000", 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022682400000000001 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.894227+0000", 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043196129999999999 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.682 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:22206 3.19 3:9db5e545:::rbd_header.116130782be1:head [watch reconnect cookie 140573340143216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e160)", 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.794205644999998, 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1078999330000001, 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22206 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:48.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787255+0000", 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787256+0000", 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6700000000000003e-07 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787260+0000", 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5039999999999998e-06 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787269+0000", 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410000000000001e-06 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787278+0000", 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1500000000000005e-06 2024-09-14T07:11:48.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787507+0000", 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022899800000000001 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:52.787530+0000", 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2999000000000001e-05 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.890926+0000", 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1033960199999999 2024-09-14T07:11:48.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.891265+0000", 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033879500000000002 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:53.895155+0000", 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003890559 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:48.687 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:48.688 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:48.689 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:48.690 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:48.685+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:48.785 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T07:11:48.785 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-09-14T07:11:48.790 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:48.785+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:48.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T07:11:48.876 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:48.877 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:48.877 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:48.877 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:48.897 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:48.915 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:48.913+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:48.991 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:48.989+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:49.014 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:49.014 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:49.015 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:49.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:49.115 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:49.113+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:49.194 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:49.189+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:49.203 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 211.90602956199999, 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:49.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:49.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:49.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 199.04655397499999, 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:49.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.214 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 199.04645169, 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:49.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:49.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:49.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 198.347399849, 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:49.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:49.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.224 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.86238828800001, 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.225 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:49.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.230 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.862354219, 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:49.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:49.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:49.234 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.86231087300001, 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:49.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.86227735, 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:49.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.242 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.8622301, 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.243 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:49.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:49.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.86211585199999, 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.247 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:49.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.862006157, 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.251 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 162.861940014, 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.255 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 106.626864166, 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:49.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:49.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:49.264 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 97.340150730000005, 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:49.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:49.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.724135688000004, 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:49.271 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:49.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.724071988999995, 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.277 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:49.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:49.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.283 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.724028654999998, 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:49.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:49.289 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.723971775999999, 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:49.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:49.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:49.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:49.295 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.723905281, 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:49.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:49.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 93.721020944000003, 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:49.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:49.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:49.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:49.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:49.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:49.308 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:49.309 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:49.310 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:49.311 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:49.305+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:49.411 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:49.409+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:49.493 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:49.493 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:49.494 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:49.494 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:49.509 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:49.511 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:49.505+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:49.612 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:49.609+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:49.632 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:49.632 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:49.646 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:49.658 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:49.675 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:49.675 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:49.691 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:49.713 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:49.713 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:49.709+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:49.730 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:49.731 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:49.749 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:49.812 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:49.812 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:49.814 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:49.809+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:49.814 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T07:11:49.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:49.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T07:11:49.931 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:49.929+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:49.956 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:49.956 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:49.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:50.059 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:50.057+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:50.120 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:50.113+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:50.136 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:50.136 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:50.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:50.219 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:50.213+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:50.319 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:50.320 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:50.320 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:50.317+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:50.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:50.421 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:50.417+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:50.462 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:50.462 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:50.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:50.521 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:50.517+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:50.622 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:50.617+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:50.644 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:50.644 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:50.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:50.725 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:50.721+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:50.779 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:50.779 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:50.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T07:11:50.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:11:50.826 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:50.821+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:50.848 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T07:11:50.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:11:50.927 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:50.921+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:50.936 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:50.936 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:50.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:51.039 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:51.037+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:51.114 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:51.114 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:51.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:51.129 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:51.125+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:51.198 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:51.199 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:11:51.217 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-14T07:11:50.818466+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":131036208,"num_objects":356,"num_object_clones":14,"num_object_copies":712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":356,"num_whiteouts":258,"num_read":340632,"num_read_kb":1115628,"num_write":21932,"num_write_kb":331476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19270,"ondisk_log_size":19270,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":347548,"kb_used_data":19472,"kb_used_omap":35,"kb_used_meta":327836,"kb_avail":565883492,"statfs":{"total":579820584960,"available":579464695808,"internally_reserved":0,"allocated":19939328,"data_stored":39178518,"data_compressed":1664109,"data_compressed_allocated":3481600,"data_compressed_original":32235520,"omap_allocated":36329,"internal_metadata":335704599},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-246909837,"num_objects":-116,"num_object_clones":-26,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":-21,"num_read":579,"num_read_kb":499,"num_write":156,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":1123856,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023816"},"pg_stats":[{"pgid":"3.1f","version":"211'380","reported_seq":6929,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T07:11:45.797558+0000","last_change":"2024-09-14T07:11:44.805566+0000","last_active":"2024-09-14T07:11:45.797558+0000","last_peered":"2024-09-14T07:11:45.797558+0000","last_clean":"2024-09-14T07:11:45.797558+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:11:45.797558+0000","last_undegraded":"2024-09-14T07:11:45.797558+0000","last_fullsized":"2024-09-14T07:11:45.797558+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:10:22.397525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.012641136000000001,"stat_sum":{"num_bytes":180240,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":6401,"num_read_kb":50527,"num_write":554,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.1e","version":"217'720","reported_seq":12973,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001526+0000","last_change":"2024-09-14T07:11:49.832703+0000","last_active":"2024-09-14T07:11:50.001526+0000","last_peered":"2024-09-14T07:11:50.001526+0000","last_clean":"2024-09-14T07:11:50.001526+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:11:50.001526+0000","last_undegraded":"2024-09-14T07:11:50.001526+0000","last_fullsized":"2024-09-14T07:11:50.001526+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:57:12.557671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077816990000000004,"stat_sum":{"num_bytes":5972628,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":14175,"num_read_kb":59693,"num_write":943,"num_write_kb":21289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1d","version":"217'1602","reported_seq":16581,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005381+0000","last_change":"2024-09-14T07:11:49.834031+0000","last_active":"2024-09-14T07:11:50.005381+0000","last_peered":"2024-09-14T07:11:50.005381+0000","last_clean":"2024-09-14T07:11:50.005381+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:11:50.005381+0000","last_undegraded":"2024-09-14T07:11:50.005381+0000","last_fullsized":"2024-09-14T07:11:50.005381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1602,"log_dups_size":0,"ondisk_log_size":1602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:22:12.091166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072780320000000003,"stat_sum":{"num_bytes":4808704,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":20893,"num_read_kb":36306,"num_write":1696,"num_write_kb":5676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1c","version":"217'501","reported_seq":20791,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.895045+0000","last_change":"2024-09-14T07:11:49.826028+0000","last_active":"2024-09-14T07:11:49.895045+0000","last_peered":"2024-09-14T07:11:49.895045+0000","last_clean":"2024-09-14T07:11:49.895045+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:11:49.895045+0000","last_undegraded":"2024-09-14T07:11:49.895045+0000","last_fullsized":"2024-09-14T07:11:49.895045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:25.506830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001223399,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21030,"num_read_kb":46630,"num_write":501,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1b","version":"210'563","reported_seq":8639,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T07:11:45.797371+0000","last_change":"2024-09-14T07:11:44.793893+0000","last_active":"2024-09-14T07:11:45.797371+0000","last_peered":"2024-09-14T07:11:45.797371+0000","last_clean":"2024-09-14T07:11:45.797371+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:11:45.797371+0000","last_undegraded":"2024-09-14T07:11:45.797371+0000","last_fullsized":"2024-09-14T07:11:45.797371+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:02:16.625709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035914660000000002,"stat_sum":{"num_bytes":20820628,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":8775,"num_read_kb":22847,"num_write":712,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.1a","version":"213'389","reported_seq":30287,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.884017+0000","last_change":"2024-09-14T07:11:46.814896+0000","last_active":"2024-09-14T07:11:46.884017+0000","last_peered":"2024-09-14T07:11:46.884017+0000","last_clean":"2024-09-14T07:11:46.884017+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:46.884017+0000","last_undegraded":"2024-09-14T07:11:46.884017+0000","last_fullsized":"2024-09-14T07:11:46.884017+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:22.158079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010132499999999999,"stat_sum":{"num_bytes":786451,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":29642,"num_read_kb":50524,"num_write":413,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"3.19","version":"217'1338","reported_seq":10587,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822350+0000","last_change":"2024-09-14T07:11:49.822350+0000","last_active":"2024-09-14T07:11:49.822350+0000","last_peered":"2024-09-14T07:11:49.822350+0000","last_clean":"2024-09-14T07:11:49.822350+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:11:49.822350+0000","last_undegraded":"2024-09-14T07:11:49.822350+0000","last_fullsized":"2024-09-14T07:11:49.822350+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1338,"log_dups_size":0,"ondisk_log_size":1338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:39.514580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042770159999999998,"stat_sum":{"num_bytes":2836518,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":14561,"num_read_kb":35933,"num_write":2117,"num_write_kb":9607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.18","version":"213'425","reported_seq":5388,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T07:11:45.797403+0000","last_change":"2024-09-14T07:11:44.805124+0000","last_active":"2024-09-14T07:11:45.797403+0000","last_peered":"2024-09-14T07:11:45.797403+0000","last_clean":"2024-09-14T07:11:45.797403+0000","last_became_active":"2024-09-14T07:11:42.792094+0000","last_became_peered":"2024-09-14T07:11:42.792094+0000","last_unstale":"2024-09-14T07:11:45.797403+0000","last_undegraded":"2024-09-14T07:11:45.797403+0000","last_fullsized":"2024-09-14T07:11:45.797403+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:12:46.314634+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01440169,"stat_sum":{"num_bytes":270384,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":4855,"num_read_kb":15253,"num_write":483,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.17","version":"217'242","reported_seq":3834,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005660+0000","last_change":"2024-09-14T07:11:49.834634+0000","last_active":"2024-09-14T07:11:50.005660+0000","last_peered":"2024-09-14T07:11:50.005660+0000","last_clean":"2024-09-14T07:11:50.005660+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:11:50.005660+0000","last_undegraded":"2024-09-14T07:11:50.005660+0000","last_fullsized":"2024-09-14T07:11:50.005660+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:55.293205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.16","version":"216'640","reported_seq":9998,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.573145+0000","last_change":"2024-09-14T07:11:50.307883+0000","last_active":"2024-09-14T07:11:50.307883+0000","last_peered":"2024-09-14T07:11:49.573145+0000","last_clean":"2024-09-14T07:11:43.788063+0000","last_became_active":"2024-09-14T07:11:48.815066+0000","last_became_peered":"2024-09-14T07:11:48.815066+0000","last_unstale":"2024-09-14T07:11:49.573145+0000","last_undegraded":"2024-09-14T07:11:49.573145+0000","last_fullsized":"2024-09-14T07:11:49.573145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084910960000000001,"stat_sum":{"num_bytes":1368064,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":738,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.15","version":"217'509","reported_seq":6975,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.833831+0000","last_change":"2024-09-14T07:11:49.833831+0000","last_active":"2024-09-14T07:11:49.833831+0000","last_peered":"2024-09-14T07:11:49.833831+0000","last_clean":"2024-09-14T07:11:49.833831+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:11:49.833831+0000","last_undegraded":"2024-09-14T07:11:49.833831+0000","last_fullsized":"2024-09-14T07:11:49.833831+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:17.752405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075586000000000004,"stat_sum":{"num_bytes":4245041,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":7401,"num_read_kb":47217,"num_write":522,"num_write_kb":16662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.14","version":"217'750","reported_seq":11076,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.933900+0000","last_change":"2024-09-14T07:11:49.830868+0000","last_active":"2024-09-14T07:11:49.933900+0000","last_peered":"2024-09-14T07:11:49.933900+0000","last_clean":"2024-09-14T07:11:49.933900+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:11:49.933900+0000","last_undegraded":"2024-09-14T07:11:49.933900+0000","last_fullsized":"2024-09-14T07:11:49.933900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:21.610420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066874480000000004,"stat_sum":{"num_bytes":12418560,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":12254,"num_read_kb":39191,"num_write":924,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.7","version":"0'0","reported_seq":429,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826029+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:11:49.826029+0000","last_peered":"2024-09-14T07:11:49.826029+0000","last_clean":"2024-09-14T07:11:49.826029+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:11:49.826029+0000","last_undegraded":"2024-09-14T07:11:49.826029+0000","last_fullsized":"2024-09-14T07:11:49.826029+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"213'689","reported_seq":12763,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T07:11:45.835908+0000","last_change":"2024-09-14T07:11:45.836152+0000","last_active":"2024-09-14T07:11:45.835908+0000","last_peered":"2024-09-14T07:11:45.835908+0000","last_clean":"2024-09-14T07:11:45.835908+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:11:45.835908+0000","last_undegraded":"2024-09-14T07:11:45.835908+0000","last_fullsized":"2024-09-14T07:11:45.835908+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:52.553869+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.034009635000000003,"stat_sum":{"num_bytes":129628,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":14012,"num_read_kb":35141,"num_write":738,"num_write_kb":5113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.819091+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:11:49.819091+0000","last_peered":"2024-09-14T07:11:49.819091+0000","last_clean":"2024-09-14T07:11:49.819091+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:11:49.819091+0000","last_undegraded":"2024-09-14T07:11:49.819091+0000","last_fullsized":"2024-09-14T07:11:49.819091+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"217'658","reported_seq":8317,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001581+0000","last_change":"2024-09-14T07:11:49.820591+0000","last_active":"2024-09-14T07:11:50.001581+0000","last_peered":"2024-09-14T07:11:50.001581+0000","last_clean":"2024-09-14T07:11:50.001581+0000","last_became_active":"2024-09-14T07:10:57.225048+0000","last_became_peered":"2024-09-14T07:10:57.225048+0000","last_unstale":"2024-09-14T07:11:50.001581+0000","last_undegraded":"2024-09-14T07:11:50.001581+0000","last_fullsized":"2024-09-14T07:11:50.001581+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:30:34.825320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025417880000000001,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":701,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.5","version":"0'0","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.827578+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:11:49.827578+0000","last_peered":"2024-09-14T07:11:49.827578+0000","last_clean":"2024-09-14T07:11:49.827578+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:11:49.827578+0000","last_undegraded":"2024-09-14T07:11:49.827578+0000","last_fullsized":"2024-09-14T07:11:49.827578+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"216'686","reported_seq":6671,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.821973+0000","last_change":"2024-09-14T07:11:49.822694+0000","last_active":"2024-09-14T07:11:49.821973+0000","last_peered":"2024-09-14T07:11:49.821973+0000","last_clean":"2024-09-14T07:11:49.821973+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:11:49.821973+0000","last_undegraded":"2024-09-14T07:11:49.821973+0000","last_fullsized":"2024-09-14T07:11:49.821973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:34:36.123624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042077470000000004,"stat_sum":{"num_bytes":131072,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":862,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.818494+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:11:49.818494+0000","last_peered":"2024-09-14T07:11:49.818494+0000","last_clean":"2024-09-14T07:11:49.818494+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:11:49.818494+0000","last_undegraded":"2024-09-14T07:11:49.818494+0000","last_fullsized":"2024-09-14T07:11:49.818494+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"217'750","reported_seq":4024,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.994308+0000","last_change":"2024-09-14T07:11:49.834054+0000","last_active":"2024-09-14T07:11:49.994308+0000","last_peered":"2024-09-14T07:11:49.994308+0000","last_clean":"2024-09-14T07:11:49.994308+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:11:49.994308+0000","last_undegraded":"2024-09-14T07:11:49.994308+0000","last_fullsized":"2024-09-14T07:11:49.994308+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:44:07.647554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006901735,"stat_sum":{"num_bytes":5210128,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":5230,"num_read_kb":19720,"num_write":1110,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.3","version":"214'489","reported_seq":10033,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.927980+0000","last_change":"2024-09-14T07:11:46.810220+0000","last_active":"2024-09-14T07:11:46.927980+0000","last_peered":"2024-09-14T07:11:46.927980+0000","last_clean":"2024-09-14T07:11:46.927980+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:11:46.927980+0000","last_undegraded":"2024-09-14T07:11:46.927980+0000","last_fullsized":"2024-09-14T07:11:46.927980+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:40:26.998349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055825800000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8723,"num_read_kb":48182,"num_write":528,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"2.2","version":"19'2","reported_seq":440,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826271+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:11:49.826271+0000","last_peered":"2024-09-14T07:11:49.826271+0000","last_clean":"2024-09-14T07:11:49.826271+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:11:49.826271+0000","last_undegraded":"2024-09-14T07:11:49.826271+0000","last_fullsized":"2024-09-14T07:11:49.826271+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"217'500","reported_seq":6349,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993429+0000","last_change":"2024-09-14T07:11:49.823563+0000","last_active":"2024-09-14T07:11:49.993429+0000","last_peered":"2024-09-14T07:11:49.993429+0000","last_clean":"2024-09-14T07:11:49.993429+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:11:49.993429+0000","last_undegraded":"2024-09-14T07:11:49.993429+0000","last_fullsized":"2024-09-14T07:11:49.993429+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:33.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042663839999999998,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7616,"num_read_kb":23803,"num_write":401,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.817757+0000","last_change":"2024-09-14T07:10:52.793359+0000","last_active":"2024-09-14T07:11:49.817757+0000","last_peered":"2024-09-14T07:11:49.817757+0000","last_clean":"2024-09-14T07:11:49.817757+0000","last_became_active":"2024-09-14T07:10:52.792887+0000","last_became_peered":"2024-09-14T07:10:52.792887+0000","last_unstale":"2024-09-14T07:11:49.817757+0000","last_undegraded":"2024-09-14T07:11:49.817757+0000","last_fullsized":"2024-09-14T07:11:49.817757+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:11.029506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"211'483","reported_seq":12962,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T07:11:45.797274+0000","last_change":"2024-09-14T07:11:44.802191+0000","last_active":"2024-09-14T07:11:45.797274+0000","last_peered":"2024-09-14T07:11:45.797274+0000","last_clean":"2024-09-14T07:11:45.797274+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:11:45.797274+0000","last_undegraded":"2024-09-14T07:11:45.797274+0000","last_fullsized":"2024-09-14T07:11:45.797274+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:14:43.170871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011789918999999999,"stat_sum":{"num_bytes":491520,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":382,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"2.0","version":"0'0","reported_seq":430,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822692+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:11:49.822692+0000","last_peered":"2024-09-14T07:11:49.822692+0000","last_clean":"2024-09-14T07:11:49.822692+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:11:49.822692+0000","last_undegraded":"2024-09-14T07:11:49.822692+0000","last_fullsized":"2024-09-14T07:11:49.822692+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"213'396","reported_seq":7828,"reported_epoch":216,"state":"peering","last_fresh":"2024-09-14T07:11:48.983941+0000","last_change":"2024-09-14T07:11:49.818862+0000","last_active":"2024-09-14T07:11:49.818862+0000","last_peered":"2024-09-14T07:11:48.983941+0000","last_clean":"2024-09-14T07:11:46.103525+0000","last_became_active":"2024-09-14T07:11:48.811974+0000","last_became_peered":"2024-09-14T07:11:48.811974+0000","last_unstale":"2024-09-14T07:11:48.983941+0000","last_undegraded":"2024-09-14T07:11:48.983941+0000","last_fullsized":"2024-09-14T07:11:48.794907+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01901893,"stat_sum":{"num_bytes":858640,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":6436,"num_read_kb":16010,"num_write":342,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"2.3","version":"17'1","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.825168+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:11:49.825168+0000","last_peered":"2024-09-14T07:11:49.825168+0000","last_clean":"2024-09-14T07:11:49.825168+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:11:49.825168+0000","last_undegraded":"2024-09-14T07:11:49.825168+0000","last_fullsized":"2024-09-14T07:11:49.825168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":542,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822949+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:11:49.822949+0000","last_peered":"2024-09-14T07:11:49.822949+0000","last_clean":"2024-09-14T07:11:49.822949+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:11:49.822949+0000","last_undegraded":"2024-09-14T07:11:49.822949+0000","last_fullsized":"2024-09-14T07:11:49.822949+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"217'535","reported_seq":10138,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.820793+0000","last_change":"2024-09-14T07:11:49.820793+0000","last_active":"2024-09-14T07:11:49.820793+0000","last_peered":"2024-09-14T07:11:49.820793+0000","last_clean":"2024-09-14T07:11:49.820793+0000","last_became_active":"2024-09-14T07:10:58.788140+0000","last_became_peered":"2024-09-14T07:10:58.788140+0000","last_unstale":"2024-09-14T07:11:49.820793+0000","last_undegraded":"2024-09-14T07:11:49.820793+0000","last_fullsized":"2024-09-14T07:11:49.820793+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:21:00.558281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002853487,"stat_sum":{"num_bytes":7600655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":9538,"num_read_kb":69033,"num_write":685,"num_write_kb":27473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.9","version":"214'740","reported_seq":8333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.824036+0000","last_change":"2024-09-14T07:11:46.827725+0000","last_active":"2024-09-14T07:11:46.824036+0000","last_peered":"2024-09-14T07:11:46.824036+0000","last_clean":"2024-09-14T07:11:46.824036+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:11:46.824036+0000","last_undegraded":"2024-09-14T07:11:46.824036+0000","last_fullsized":"2024-09-14T07:11:46.824036+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:43.003401+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021946032000000001,"stat_sum":{"num_bytes":568040,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":10,"num_read":10159,"num_read_kb":48108,"num_write":750,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"3.a","version":"217'437","reported_seq":8196,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.835504+0000","last_change":"2024-09-14T07:11:49.835504+0000","last_active":"2024-09-14T07:11:49.835504+0000","last_peered":"2024-09-14T07:11:49.835504+0000","last_clean":"2024-09-14T07:11:49.835504+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:11:49.835504+0000","last_undegraded":"2024-09-14T07:11:49.835504+0000","last_fullsized":"2024-09-14T07:11:49.835504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:24.697088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047041990000000001,"stat_sum":{"num_bytes":1452032,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":7826,"num_read_kb":35943,"num_write":413,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.b","version":"217'1019","reported_seq":8107,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.985806+0000","last_change":"2024-09-14T07:11:49.821839+0000","last_active":"2024-09-14T07:11:49.985806+0000","last_peered":"2024-09-14T07:11:49.985806+0000","last_clean":"2024-09-14T07:11:49.985806+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:11:49.985806+0000","last_undegraded":"2024-09-14T07:11:49.985806+0000","last_fullsized":"2024-09-14T07:11:49.985806+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:20:17.413757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036922539999999998,"stat_sum":{"num_bytes":393384,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":10228,"num_read_kb":41579,"num_write":1377,"num_write_kb":13967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.c","version":"217'547","reported_seq":15456,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830180+0000","last_change":"2024-09-14T07:11:49.830180+0000","last_active":"2024-09-14T07:11:49.830180+0000","last_peered":"2024-09-14T07:11:49.830180+0000","last_clean":"2024-09-14T07:11:49.830180+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:49.830180+0000","last_undegraded":"2024-09-14T07:11:49.830180+0000","last_fullsized":"2024-09-14T07:11:49.830180+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:03.498897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039997849999999996,"stat_sum":{"num_bytes":262144,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":463,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.d","version":"217'342","reported_seq":4125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993312+0000","last_change":"2024-09-14T07:11:49.834171+0000","last_active":"2024-09-14T07:11:49.993312+0000","last_peered":"2024-09-14T07:11:49.993312+0000","last_clean":"2024-09-14T07:11:49.993312+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:11:49.993312+0000","last_undegraded":"2024-09-14T07:11:49.993312+0000","last_fullsized":"2024-09-14T07:11:49.993312+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:13.447198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071541449999999998,"stat_sum":{"num_bytes":1048576,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":323,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.e","version":"216'443","reported_seq":17414,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.304818+0000","last_change":"2024-09-14T07:11:50.307914+0000","last_active":"2024-09-14T07:11:50.307914+0000","last_peered":"2024-09-14T07:11:49.304818+0000","last_clean":"2024-09-14T07:11:40.083699+0000","last_became_active":"2024-09-14T07:11:48.814324+0000","last_became_peered":"2024-09-14T07:11:48.814324+0000","last_unstale":"2024-09-14T07:11:49.304818+0000","last_undegraded":"2024-09-14T07:11:49.304818+0000","last_fullsized":"2024-09-14T07:11:49.304818+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0016376439999999999,"stat_sum":{"num_bytes":524288,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":438,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.f","version":"217'484","reported_seq":8518,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830984+0000","last_change":"2024-09-14T07:11:49.830984+0000","last_active":"2024-09-14T07:11:49.830984+0000","last_peered":"2024-09-14T07:11:49.830984+0000","last_clean":"2024-09-14T07:11:49.830984+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:11:49.830984+0000","last_undegraded":"2024-09-14T07:11:49.830984+0000","last_fullsized":"2024-09-14T07:11:49.830984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:49:26.120361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004728945,"stat_sum":{"num_bytes":131072,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":615,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.10","version":"216'462","reported_seq":8117,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.138923+0000","last_change":"2024-09-14T07:11:50.307757+0000","last_active":"2024-09-14T07:11:50.307757+0000","last_peered":"2024-09-14T07:11:49.138923+0000","last_clean":"2024-09-14T07:11:45.797283+0000","last_became_active":"2024-09-14T07:11:48.810645+0000","last_became_peered":"2024-09-14T07:11:48.810645+0000","last_unstale":"2024-09-14T07:11:49.138923+0000","last_undegraded":"2024-09-14T07:11:49.138923+0000","last_fullsized":"2024-09-14T07:11:48.791074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023065630000000002,"stat_sum":{"num_bytes":7864320,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":651,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.11","version":"217'537","reported_seq":9421,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826638+0000","last_change":"2024-09-14T07:11:49.826638+0000","last_active":"2024-09-14T07:11:49.826638+0000","last_peered":"2024-09-14T07:11:49.826638+0000","last_clean":"2024-09-14T07:11:49.826638+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:11:49.826638+0000","last_undegraded":"2024-09-14T07:11:49.826638+0000","last_fullsized":"2024-09-14T07:11:49.826638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:22:42.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036882769999999998,"stat_sum":{"num_bytes":17920512,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":10491,"num_read_kb":24428,"num_write":419,"num_write_kb":9181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.12","version":"217'558","reported_seq":7288,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.997627+0000","last_change":"2024-09-14T07:11:49.835043+0000","last_active":"2024-09-14T07:11:49.997627+0000","last_peered":"2024-09-14T07:11:49.997627+0000","last_clean":"2024-09-14T07:11:49.997627+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:11:49.997627+0000","last_undegraded":"2024-09-14T07:11:49.997627+0000","last_fullsized":"2024-09-14T07:11:49.997627+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:25:26.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005177877,"stat_sum":{"num_bytes":18746368,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":428,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.13","version":"217'381","reported_seq":4946,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.831753+0000","last_change":"2024-09-14T07:11:49.831753+0000","last_active":"2024-09-14T07:11:49.831753+0000","last_peered":"2024-09-14T07:11:49.831753+0000","last_clean":"2024-09-14T07:11:49.831753+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:11:49.831753+0000","last_undegraded":"2024-09-14T07:11:49.831753+0000","last_fullsized":"2024-09-14T07:11:49.831753+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:29.134495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073722479999999996,"stat_sum":{"num_bytes":9211904,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":356,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":130445821,"num_objects":352,"num_object_clones":14,"num_object_copies":704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":258,"num_read":340566,"num_read_kb":1115573,"num_write":21829,"num_write_kb":329846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2883584,"data_stored":16195098,"data_compressed":1415166,"data_compressed_allocated":2158592,"data_compressed_original":15826944,"omap_allocated":12989,"internal_metadata":0},"log_size":19195,"ondisk_log_size":19195,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78684,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293156,"statfs":{"total":96636764160,"available":96556191744,"internally_reserved":0,"allocated":3145728,"data_stored":3831785,"data_compressed":256797,"data_compressed_allocated":425984,"data_compressed_original":2682880,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607608,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52124,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319716,"statfs":{"total":96636764160,"available":96583389184,"internally_reserved":0,"allocated":2957312,"data_stored":3660859,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607608,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70536,"kb_used_data":3196,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301304,"statfs":{"total":96636764160,"available":96564535296,"internally_reserved":0,"allocated":3272704,"data_stored":4258061,"data_compressed":351328,"data_compressed_allocated":512000,"data_compressed_original":3088384,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607609,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64988,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306852,"statfs":{"total":96636764160,"available":96570216448,"internally_reserved":0,"allocated":3330048,"data_stored":6111343,"data_compressed":270417,"data_compressed_allocated":561152,"data_compressed_original":4960256,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607609,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48052,"kb_used_data":2980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323788,"statfs":{"total":96636764160,"available":96587558912,"internally_reserved":0,"allocated":3051520,"data_stored":4216320,"data_compressed":171212,"data_compressed_allocated":348160,"data_compressed_original":3092480,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164866,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":4084,"kb_used_omap":27,"kb_used_meta":29028,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":4182016,"data_stored":17100150,"data_compressed":467138,"data_compressed_allocated":1388544,"data_compressed_original":15892480,"omap_allocated":28356,"internal_metadata":29724988},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":696320,"data_stored":5206044,"data_compressed":301593,"data_compressed_allocated":544768,"data_compressed_original":5120000,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2034009,"data_compressed":161836,"data_compressed_allocated":258048,"data_compressed_original":2002944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":1396187,"data_compressed":231909,"data_compressed_allocated":282624,"data_compressed_original":1343488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":2582245,"data_compressed":345270,"data_compressed_allocated":475136,"data_compressed_original":2514944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":487424,"data_stored":2401221,"data_compressed":227341,"data_compressed_allocated":352256,"data_compressed_original":2326528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2575392,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:11:51.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:11:51.259 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:51.257+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:51.290 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:51.290 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:51.305 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:51.352 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:51.349+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:51.430 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:51.425+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:51.436 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:51.436 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:51.451 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:51.536 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:51.533+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:51.551 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:51.552 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:11:51.568 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-14T07:11:51.270738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128717872,"num_objects":345,"num_object_clones":10,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":255,"num_read":340638,"num_read_kb":1115632,"num_write":21938,"num_write_kb":331477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19287,"ondisk_log_size":19287,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":350588,"kb_used_data":18336,"kb_used_omap":35,"kb_used_meta":331996,"kb_avail":565880452,"statfs":{"total":579820584960,"available":579461582848,"internally_reserved":0,"allocated":18776064,"data_stored":26346232,"data_compressed":1453205,"data_compressed_allocated":2420736,"data_compressed_original":19505152,"omap_allocated":36329,"internal_metadata":339964439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-249228173,"num_objects":-127,"num_object_clones":-30,"num_object_copies":-254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-127,"num_whiteouts":-24,"num_read":585,"num_read_kb":503,"num_write":162,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":1123856,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001228"},"pg_stats":[{"pgid":"3.1f","version":"217'381","reported_seq":6950,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820211+0000","last_change":"2024-09-14T07:11:49.823797+0000","last_active":"2024-09-14T07:11:50.820211+0000","last_peered":"2024-09-14T07:11:50.820211+0000","last_clean":"2024-09-14T07:11:50.820211+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:11:50.820211+0000","last_undegraded":"2024-09-14T07:11:50.820211+0000","last_fullsized":"2024-09-14T07:11:50.820211+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:04:11.679482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081229,"stat_sum":{"num_bytes":16,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6401,"num_read_kb":50527,"num_write":555,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1e","version":"217'720","reported_seq":12973,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001526+0000","last_change":"2024-09-14T07:11:49.832703+0000","last_active":"2024-09-14T07:11:50.001526+0000","last_peered":"2024-09-14T07:11:50.001526+0000","last_clean":"2024-09-14T07:11:50.001526+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:11:50.001526+0000","last_undegraded":"2024-09-14T07:11:50.001526+0000","last_fullsized":"2024-09-14T07:11:50.001526+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:57:12.557671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077816990000000004,"stat_sum":{"num_bytes":5972628,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":14175,"num_read_kb":59693,"num_write":943,"num_write_kb":21289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1d","version":"217'1602","reported_seq":16581,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005381+0000","last_change":"2024-09-14T07:11:49.834031+0000","last_active":"2024-09-14T07:11:50.005381+0000","last_peered":"2024-09-14T07:11:50.005381+0000","last_clean":"2024-09-14T07:11:50.005381+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:11:50.005381+0000","last_undegraded":"2024-09-14T07:11:50.005381+0000","last_fullsized":"2024-09-14T07:11:50.005381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1602,"log_dups_size":0,"ondisk_log_size":1602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:22:12.091166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072780320000000003,"stat_sum":{"num_bytes":4808704,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":20893,"num_read_kb":36306,"num_write":1696,"num_write_kb":5676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1c","version":"217'501","reported_seq":20791,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.895045+0000","last_change":"2024-09-14T07:11:49.826028+0000","last_active":"2024-09-14T07:11:49.895045+0000","last_peered":"2024-09-14T07:11:49.895045+0000","last_clean":"2024-09-14T07:11:49.895045+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:11:49.895045+0000","last_undegraded":"2024-09-14T07:11:49.895045+0000","last_fullsized":"2024-09-14T07:11:49.895045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:25.506830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001223399,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21030,"num_read_kb":46630,"num_write":501,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1b","version":"214'572","reported_seq":8666,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819776+0000","last_change":"2024-09-14T07:11:49.823068+0000","last_active":"2024-09-14T07:11:50.819776+0000","last_peered":"2024-09-14T07:11:50.819776+0000","last_clean":"2024-09-14T07:11:50.819776+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:11:50.819776+0000","last_undegraded":"2024-09-14T07:11:50.819776+0000","last_fullsized":"2024-09-14T07:11:50.819776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:06:03.189849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015995510000000001,"stat_sum":{"num_bytes":19174036,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":8775,"num_read_kb":22847,"num_write":713,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1a","version":"213'389","reported_seq":30287,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.884017+0000","last_change":"2024-09-14T07:11:46.814896+0000","last_active":"2024-09-14T07:11:46.884017+0000","last_peered":"2024-09-14T07:11:46.884017+0000","last_clean":"2024-09-14T07:11:46.884017+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:46.884017+0000","last_undegraded":"2024-09-14T07:11:46.884017+0000","last_fullsized":"2024-09-14T07:11:46.884017+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:22.158079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010132499999999999,"stat_sum":{"num_bytes":786451,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":29642,"num_read_kb":50524,"num_write":413,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"3.19","version":"217'1338","reported_seq":10587,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822350+0000","last_change":"2024-09-14T07:11:49.822350+0000","last_active":"2024-09-14T07:11:49.822350+0000","last_peered":"2024-09-14T07:11:49.822350+0000","last_clean":"2024-09-14T07:11:49.822350+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:11:49.822350+0000","last_undegraded":"2024-09-14T07:11:49.822350+0000","last_fullsized":"2024-09-14T07:11:49.822350+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1338,"log_dups_size":0,"ondisk_log_size":1338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:39.514580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042770159999999998,"stat_sum":{"num_bytes":2836518,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":14561,"num_read_kb":35933,"num_write":2117,"num_write_kb":9607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.18","version":"218'427","reported_seq":5411,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.949888+0000","last_change":"2024-09-14T07:11:49.822391+0000","last_active":"2024-09-14T07:11:50.949888+0000","last_peered":"2024-09-14T07:11:50.949888+0000","last_clean":"2024-09-14T07:11:50.949888+0000","last_became_active":"2024-09-14T07:11:42.792094+0000","last_became_peered":"2024-09-14T07:11:42.792094+0000","last_unstale":"2024-09-14T07:11:50.949888+0000","last_undegraded":"2024-09-14T07:11:50.949888+0000","last_fullsized":"2024-09-14T07:11:50.949888+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:42:01.973449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096171600000000005,"stat_sum":{"num_bytes":270384,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":4861,"num_read_kb":15257,"num_write":485,"num_write_kb":15533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.17","version":"217'242","reported_seq":3834,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005660+0000","last_change":"2024-09-14T07:11:49.834634+0000","last_active":"2024-09-14T07:11:50.005660+0000","last_peered":"2024-09-14T07:11:50.005660+0000","last_clean":"2024-09-14T07:11:50.005660+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:11:50.005660+0000","last_undegraded":"2024-09-14T07:11:50.005660+0000","last_fullsized":"2024-09-14T07:11:50.005660+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:55.293205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.16","version":"216'640","reported_seq":9998,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.573145+0000","last_change":"2024-09-14T07:11:50.307883+0000","last_active":"2024-09-14T07:11:50.307883+0000","last_peered":"2024-09-14T07:11:49.573145+0000","last_clean":"2024-09-14T07:11:43.788063+0000","last_became_active":"2024-09-14T07:11:48.815066+0000","last_became_peered":"2024-09-14T07:11:48.815066+0000","last_unstale":"2024-09-14T07:11:49.573145+0000","last_undegraded":"2024-09-14T07:11:49.573145+0000","last_fullsized":"2024-09-14T07:11:49.573145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084910960000000001,"stat_sum":{"num_bytes":1368064,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":738,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.15","version":"217'509","reported_seq":6975,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.833831+0000","last_change":"2024-09-14T07:11:49.833831+0000","last_active":"2024-09-14T07:11:49.833831+0000","last_peered":"2024-09-14T07:11:49.833831+0000","last_clean":"2024-09-14T07:11:49.833831+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:11:49.833831+0000","last_undegraded":"2024-09-14T07:11:49.833831+0000","last_fullsized":"2024-09-14T07:11:49.833831+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:17.752405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075586000000000004,"stat_sum":{"num_bytes":4245041,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":7401,"num_read_kb":47217,"num_write":522,"num_write_kb":16662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.14","version":"217'750","reported_seq":11076,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.933900+0000","last_change":"2024-09-14T07:11:49.830868+0000","last_active":"2024-09-14T07:11:49.933900+0000","last_peered":"2024-09-14T07:11:49.933900+0000","last_clean":"2024-09-14T07:11:49.933900+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:11:49.933900+0000","last_undegraded":"2024-09-14T07:11:49.933900+0000","last_fullsized":"2024-09-14T07:11:49.933900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:21.610420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066874480000000004,"stat_sum":{"num_bytes":12418560,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":12254,"num_read_kb":39191,"num_write":924,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.7","version":"0'0","reported_seq":429,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826029+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:11:49.826029+0000","last_peered":"2024-09-14T07:11:49.826029+0000","last_clean":"2024-09-14T07:11:49.826029+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:11:49.826029+0000","last_undegraded":"2024-09-14T07:11:49.826029+0000","last_fullsized":"2024-09-14T07:11:49.826029+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"218'692","reported_seq":12786,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820180+0000","last_change":"2024-09-14T07:11:49.823661+0000","last_active":"2024-09-14T07:11:50.820180+0000","last_peered":"2024-09-14T07:11:50.820180+0000","last_clean":"2024-09-14T07:11:50.820180+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:11:50.820180+0000","last_undegraded":"2024-09-14T07:11:50.820180+0000","last_fullsized":"2024-09-14T07:11:50.820180+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:52:36.850781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001773719,"stat_sum":{"num_bytes":129628,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":14012,"num_read_kb":35141,"num_write":738,"num_write_kb":5113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.819091+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:11:49.819091+0000","last_peered":"2024-09-14T07:11:49.819091+0000","last_clean":"2024-09-14T07:11:49.819091+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:11:49.819091+0000","last_undegraded":"2024-09-14T07:11:49.819091+0000","last_fullsized":"2024-09-14T07:11:49.819091+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"217'658","reported_seq":8317,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001581+0000","last_change":"2024-09-14T07:11:49.820591+0000","last_active":"2024-09-14T07:11:50.001581+0000","last_peered":"2024-09-14T07:11:50.001581+0000","last_clean":"2024-09-14T07:11:50.001581+0000","last_became_active":"2024-09-14T07:10:57.225048+0000","last_became_peered":"2024-09-14T07:10:57.225048+0000","last_unstale":"2024-09-14T07:11:50.001581+0000","last_undegraded":"2024-09-14T07:11:50.001581+0000","last_fullsized":"2024-09-14T07:11:50.001581+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:30:34.825320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025417880000000001,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":701,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.5","version":"0'0","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.827578+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:11:49.827578+0000","last_peered":"2024-09-14T07:11:49.827578+0000","last_clean":"2024-09-14T07:11:49.827578+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:11:49.827578+0000","last_undegraded":"2024-09-14T07:11:49.827578+0000","last_fullsized":"2024-09-14T07:11:49.827578+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"216'686","reported_seq":6671,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.821973+0000","last_change":"2024-09-14T07:11:49.822694+0000","last_active":"2024-09-14T07:11:49.821973+0000","last_peered":"2024-09-14T07:11:49.821973+0000","last_clean":"2024-09-14T07:11:49.821973+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:11:49.821973+0000","last_undegraded":"2024-09-14T07:11:49.821973+0000","last_fullsized":"2024-09-14T07:11:49.821973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:34:36.123624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042077470000000004,"stat_sum":{"num_bytes":131072,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":862,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.818494+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:11:49.818494+0000","last_peered":"2024-09-14T07:11:49.818494+0000","last_clean":"2024-09-14T07:11:49.818494+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:11:49.818494+0000","last_undegraded":"2024-09-14T07:11:49.818494+0000","last_fullsized":"2024-09-14T07:11:49.818494+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"217'750","reported_seq":4024,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.994308+0000","last_change":"2024-09-14T07:11:49.834054+0000","last_active":"2024-09-14T07:11:49.994308+0000","last_peered":"2024-09-14T07:11:49.994308+0000","last_clean":"2024-09-14T07:11:49.994308+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:11:49.994308+0000","last_undegraded":"2024-09-14T07:11:49.994308+0000","last_fullsized":"2024-09-14T07:11:49.994308+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:44:07.647554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006901735,"stat_sum":{"num_bytes":5210128,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":5230,"num_read_kb":19720,"num_write":1110,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.3","version":"214'489","reported_seq":10033,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.927980+0000","last_change":"2024-09-14T07:11:46.810220+0000","last_active":"2024-09-14T07:11:46.927980+0000","last_peered":"2024-09-14T07:11:46.927980+0000","last_clean":"2024-09-14T07:11:46.927980+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:11:46.927980+0000","last_undegraded":"2024-09-14T07:11:46.927980+0000","last_fullsized":"2024-09-14T07:11:46.927980+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:40:26.998349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055825800000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8723,"num_read_kb":48182,"num_write":528,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"2.2","version":"19'2","reported_seq":440,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826271+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:11:49.826271+0000","last_peered":"2024-09-14T07:11:49.826271+0000","last_clean":"2024-09-14T07:11:49.826271+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:11:49.826271+0000","last_undegraded":"2024-09-14T07:11:49.826271+0000","last_fullsized":"2024-09-14T07:11:49.826271+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"217'500","reported_seq":6349,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993429+0000","last_change":"2024-09-14T07:11:49.823563+0000","last_active":"2024-09-14T07:11:49.993429+0000","last_peered":"2024-09-14T07:11:49.993429+0000","last_clean":"2024-09-14T07:11:49.993429+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:11:49.993429+0000","last_undegraded":"2024-09-14T07:11:49.993429+0000","last_fullsized":"2024-09-14T07:11:49.993429+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:33.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042663839999999998,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7616,"num_read_kb":23803,"num_write":401,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.817757+0000","last_change":"2024-09-14T07:10:52.793359+0000","last_active":"2024-09-14T07:11:49.817757+0000","last_peered":"2024-09-14T07:11:49.817757+0000","last_clean":"2024-09-14T07:11:49.817757+0000","last_became_active":"2024-09-14T07:10:52.792887+0000","last_became_peered":"2024-09-14T07:10:52.792887+0000","last_unstale":"2024-09-14T07:11:49.817757+0000","last_undegraded":"2024-09-14T07:11:49.817757+0000","last_fullsized":"2024-09-14T07:11:49.817757+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:11.029506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":12984,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819542+0000","last_change":"2024-09-14T07:11:49.822587+0000","last_active":"2024-09-14T07:11:50.819542+0000","last_peered":"2024-09-14T07:11:50.819542+0000","last_clean":"2024-09-14T07:11:50.819542+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:11:50.819542+0000","last_undegraded":"2024-09-14T07:11:50.819542+0000","last_fullsized":"2024-09-14T07:11:50.819542+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:12:01.776651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001171233,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.0","version":"0'0","reported_seq":430,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822692+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:11:49.822692+0000","last_peered":"2024-09-14T07:11:49.822692+0000","last_clean":"2024-09-14T07:11:49.822692+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:11:49.822692+0000","last_undegraded":"2024-09-14T07:11:49.822692+0000","last_fullsized":"2024-09-14T07:11:49.822692+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"213'396","reported_seq":7828,"reported_epoch":216,"state":"peering","last_fresh":"2024-09-14T07:11:48.983941+0000","last_change":"2024-09-14T07:11:49.818862+0000","last_active":"2024-09-14T07:11:49.818862+0000","last_peered":"2024-09-14T07:11:48.983941+0000","last_clean":"2024-09-14T07:11:46.103525+0000","last_became_active":"2024-09-14T07:11:48.811974+0000","last_became_peered":"2024-09-14T07:11:48.811974+0000","last_unstale":"2024-09-14T07:11:48.983941+0000","last_undegraded":"2024-09-14T07:11:48.983941+0000","last_fullsized":"2024-09-14T07:11:48.794907+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01901893,"stat_sum":{"num_bytes":858640,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":6436,"num_read_kb":16010,"num_write":342,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"2.3","version":"17'1","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.825168+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:11:49.825168+0000","last_peered":"2024-09-14T07:11:49.825168+0000","last_clean":"2024-09-14T07:11:49.825168+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:11:49.825168+0000","last_undegraded":"2024-09-14T07:11:49.825168+0000","last_fullsized":"2024-09-14T07:11:49.825168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":542,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822949+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:11:49.822949+0000","last_peered":"2024-09-14T07:11:49.822949+0000","last_clean":"2024-09-14T07:11:49.822949+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:11:49.822949+0000","last_undegraded":"2024-09-14T07:11:49.822949+0000","last_fullsized":"2024-09-14T07:11:49.822949+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"217'535","reported_seq":10138,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.820793+0000","last_change":"2024-09-14T07:11:49.820793+0000","last_active":"2024-09-14T07:11:49.820793+0000","last_peered":"2024-09-14T07:11:49.820793+0000","last_clean":"2024-09-14T07:11:49.820793+0000","last_became_active":"2024-09-14T07:10:58.788140+0000","last_became_peered":"2024-09-14T07:10:58.788140+0000","last_unstale":"2024-09-14T07:11:49.820793+0000","last_undegraded":"2024-09-14T07:11:49.820793+0000","last_fullsized":"2024-09-14T07:11:49.820793+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:21:00.558281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002853487,"stat_sum":{"num_bytes":7600655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":9538,"num_read_kb":69033,"num_write":685,"num_write_kb":27473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.9","version":"214'740","reported_seq":8333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.824036+0000","last_change":"2024-09-14T07:11:46.827725+0000","last_active":"2024-09-14T07:11:46.824036+0000","last_peered":"2024-09-14T07:11:46.824036+0000","last_clean":"2024-09-14T07:11:46.824036+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:11:46.824036+0000","last_undegraded":"2024-09-14T07:11:46.824036+0000","last_fullsized":"2024-09-14T07:11:46.824036+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:43.003401+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021946032000000001,"stat_sum":{"num_bytes":568040,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":10,"num_read":10159,"num_read_kb":48108,"num_write":750,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"3.a","version":"217'437","reported_seq":8196,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.835504+0000","last_change":"2024-09-14T07:11:49.835504+0000","last_active":"2024-09-14T07:11:49.835504+0000","last_peered":"2024-09-14T07:11:49.835504+0000","last_clean":"2024-09-14T07:11:49.835504+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:11:49.835504+0000","last_undegraded":"2024-09-14T07:11:49.835504+0000","last_fullsized":"2024-09-14T07:11:49.835504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:24.697088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047041990000000001,"stat_sum":{"num_bytes":1452032,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":7826,"num_read_kb":35943,"num_write":413,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.b","version":"217'1019","reported_seq":8107,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.985806+0000","last_change":"2024-09-14T07:11:49.821839+0000","last_active":"2024-09-14T07:11:49.985806+0000","last_peered":"2024-09-14T07:11:49.985806+0000","last_clean":"2024-09-14T07:11:49.985806+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:11:49.985806+0000","last_undegraded":"2024-09-14T07:11:49.985806+0000","last_fullsized":"2024-09-14T07:11:49.985806+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:20:17.413757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036922539999999998,"stat_sum":{"num_bytes":393384,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":10228,"num_read_kb":41579,"num_write":1377,"num_write_kb":13967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.c","version":"217'547","reported_seq":15456,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830180+0000","last_change":"2024-09-14T07:11:49.830180+0000","last_active":"2024-09-14T07:11:49.830180+0000","last_peered":"2024-09-14T07:11:49.830180+0000","last_clean":"2024-09-14T07:11:49.830180+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:49.830180+0000","last_undegraded":"2024-09-14T07:11:49.830180+0000","last_fullsized":"2024-09-14T07:11:49.830180+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:03.498897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039997849999999996,"stat_sum":{"num_bytes":262144,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":463,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.d","version":"217'342","reported_seq":4125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993312+0000","last_change":"2024-09-14T07:11:49.834171+0000","last_active":"2024-09-14T07:11:49.993312+0000","last_peered":"2024-09-14T07:11:49.993312+0000","last_clean":"2024-09-14T07:11:49.993312+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:11:49.993312+0000","last_undegraded":"2024-09-14T07:11:49.993312+0000","last_fullsized":"2024-09-14T07:11:49.993312+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:13.447198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071541449999999998,"stat_sum":{"num_bytes":1048576,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":323,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.e","version":"216'443","reported_seq":17414,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.304818+0000","last_change":"2024-09-14T07:11:50.307914+0000","last_active":"2024-09-14T07:11:50.307914+0000","last_peered":"2024-09-14T07:11:49.304818+0000","last_clean":"2024-09-14T07:11:40.083699+0000","last_became_active":"2024-09-14T07:11:48.814324+0000","last_became_peered":"2024-09-14T07:11:48.814324+0000","last_unstale":"2024-09-14T07:11:49.304818+0000","last_undegraded":"2024-09-14T07:11:49.304818+0000","last_fullsized":"2024-09-14T07:11:49.304818+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0016376439999999999,"stat_sum":{"num_bytes":524288,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":438,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.f","version":"217'484","reported_seq":8518,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830984+0000","last_change":"2024-09-14T07:11:49.830984+0000","last_active":"2024-09-14T07:11:49.830984+0000","last_peered":"2024-09-14T07:11:49.830984+0000","last_clean":"2024-09-14T07:11:49.830984+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:11:49.830984+0000","last_undegraded":"2024-09-14T07:11:49.830984+0000","last_fullsized":"2024-09-14T07:11:49.830984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:49:26.120361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004728945,"stat_sum":{"num_bytes":131072,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":615,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.10","version":"216'462","reported_seq":8117,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.138923+0000","last_change":"2024-09-14T07:11:50.307757+0000","last_active":"2024-09-14T07:11:50.307757+0000","last_peered":"2024-09-14T07:11:49.138923+0000","last_clean":"2024-09-14T07:11:45.797283+0000","last_became_active":"2024-09-14T07:11:48.810645+0000","last_became_peered":"2024-09-14T07:11:48.810645+0000","last_unstale":"2024-09-14T07:11:49.138923+0000","last_undegraded":"2024-09-14T07:11:49.138923+0000","last_fullsized":"2024-09-14T07:11:48.791074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023065630000000002,"stat_sum":{"num_bytes":7864320,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":651,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.11","version":"217'537","reported_seq":9421,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826638+0000","last_change":"2024-09-14T07:11:49.826638+0000","last_active":"2024-09-14T07:11:49.826638+0000","last_peered":"2024-09-14T07:11:49.826638+0000","last_clean":"2024-09-14T07:11:49.826638+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:11:49.826638+0000","last_undegraded":"2024-09-14T07:11:49.826638+0000","last_fullsized":"2024-09-14T07:11:49.826638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:22:42.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036882769999999998,"stat_sum":{"num_bytes":17920512,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":10491,"num_read_kb":24428,"num_write":419,"num_write_kb":9181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.12","version":"217'558","reported_seq":7288,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.997627+0000","last_change":"2024-09-14T07:11:49.835043+0000","last_active":"2024-09-14T07:11:49.997627+0000","last_peered":"2024-09-14T07:11:49.997627+0000","last_clean":"2024-09-14T07:11:49.997627+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:11:49.997627+0000","last_undegraded":"2024-09-14T07:11:49.997627+0000","last_fullsized":"2024-09-14T07:11:49.997627+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:25:26.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005177877,"stat_sum":{"num_bytes":18746368,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":428,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.13","version":"217'381","reported_seq":4946,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.831753+0000","last_change":"2024-09-14T07:11:49.831753+0000","last_active":"2024-09-14T07:11:49.831753+0000","last_peered":"2024-09-14T07:11:49.831753+0000","last_clean":"2024-09-14T07:11:49.831753+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:11:49.831753+0000","last_undegraded":"2024-09-14T07:11:49.831753+0000","last_fullsized":"2024-09-14T07:11:49.831753+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:29.134495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073722479999999996,"stat_sum":{"num_bytes":9211904,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":356,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":128127485,"num_objects":341,"num_object_clones":10,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":255,"num_read":340572,"num_read_kb":1115577,"num_write":21835,"num_write_kb":329847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2539520,"data_stored":13576088,"data_compressed":1362915,"data_compressed_allocated":1904640,"data_compressed_original":13279232,"omap_allocated":12989,"internal_metadata":0},"log_size":19212,"ondisk_log_size":19212,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78684,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293156,"statfs":{"total":96636764160,"available":96556191744,"internally_reserved":0,"allocated":3145728,"data_stored":3831785,"data_compressed":256797,"data_compressed_allocated":425984,"data_compressed_original":2682880,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607608,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52124,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319716,"statfs":{"total":96636764160,"available":96583389184,"internally_reserved":0,"allocated":2957312,"data_stored":3660859,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607608,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70536,"kb_used_data":3196,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301304,"statfs":{"total":96636764160,"available":96564535296,"internally_reserved":0,"allocated":3272704,"data_stored":4258061,"data_compressed":351328,"data_compressed_allocated":512000,"data_compressed_original":3088384,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607609,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64988,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306852,"statfs":{"total":96636764160,"available":96570216448,"internally_reserved":0,"allocated":3330048,"data_stored":6111343,"data_compressed":270417,"data_compressed_allocated":561152,"data_compressed_original":4960256,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607609,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48052,"kb_used_data":2980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323788,"statfs":{"total":96636764160,"available":96587558912,"internally_reserved":0,"allocated":3051520,"data_stored":4216320,"data_compressed":171212,"data_compressed_allocated":348160,"data_compressed_original":3092480,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":2948,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":3018752,"data_stored":4267864,"data_compressed":256234,"data_compressed_allocated":327680,"data_compressed_original":3162112,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":2587034,"data_compressed":249342,"data_compressed_allocated":290816,"data_compressed_original":2572288,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2034009,"data_compressed":161836,"data_compressed_allocated":258048,"data_compressed_original":2002944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":1396187,"data_compressed":231909,"data_compressed_allocated":282624,"data_compressed_original":1343488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":2582245,"data_compressed":345270,"data_compressed_allocated":475136,"data_compressed_original":2514944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":487424,"data_stored":2401221,"data_compressed":227341,"data_compressed_allocated":352256,"data_compressed_original":2326528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2575392,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:11:51.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T07:11:51.614 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:51.615 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:51.629 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:51.659 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:51.657+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:51.733 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:51.729+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:51.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:11:51.795 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:51.795 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:51.809 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:51.835 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:11:51.856 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:51.853+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:51.908 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:51.908 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:51.924 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:51.934 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:51.929+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:51.941 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:51.944 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:51.944 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:51.959 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:51.960 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:51.970 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:51.970 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":273,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":219,"num_osds":6,"num_up_osds":6,"osd_up_since":1726297902,"num_in_osds":6,"osd_in_since":1726297852,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"creating+peering","count":3},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":345,"data_bytes":128717872,"bytes_used":359002112,"bytes_avail":579461582848,"bytes_total":579820584960,"inactive_pgs_ratio":0.097560971975326538,"recovering_objects_per_sec":105,"recovering_bytes_per_sec":280877,"recovering_keys_per_sec":0,"num_objects_recovered":424,"num_bytes_recovered":1123856,"num_keys_recovered":0,"read_bytes_sec":128728,"write_bytes_sec":8701,"read_op_per_sec":146,"write_op_per_sec":40},"fsmap":{"epoch":1,"btime":"2024-09-14T07:07:18:019388+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T07:08:33.220112+0000","services":{}},"progress_events":{}} 2024-09-14T07:11:51.980 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:51.989 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T07:11:51.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:11:52.000 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:52.059 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:52.057+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:52.125 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:52.125 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:52.125 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:52.125 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:52.125 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:52.135 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:52.133+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:52.139 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:52.237 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:52.233+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:52.263 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4803.0:1907 3.1a 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 5636096~1024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 214.96478464, 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093613290000000002, 2024-09-14T07:11:52.264 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4803", 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/799140076", 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1907 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296869+0000", 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.296871+0000", 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318025+0000", 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.021154393000000001 2024-09-14T07:11:52.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318057+0000", 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1228e-05 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318069+0000", 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2227e-05 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318169+0000", 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100487 2024-09-14T07:11:52.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.318323+0000", 2024-09-14T07:11:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015361899999999999 2024-09-14T07:11:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.320099+0000", 2024-09-14T07:11:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001775726 2024-09-14T07:11:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.363428+0000", 2024-09-14T07:11:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043329072000000003 2024-09-14T07:11:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390315+0000", 2024-09-14T07:11:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.026887571999999998 2024-09-14T07:11:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390433+0000", 2024-09-14T07:11:52.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000118176 2024-09-14T07:11:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:17.390482+0000", 2024-09-14T07:11:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9047e-05 2024-09-14T07:11:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.272 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4353 3.9 3.41433489 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:52.273 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 202.10530905300001, 2024-09-14T07:11:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94742959000000004, 2024-09-14T07:11:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4353 2024-09-14T07:11:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156345+0000", 2024-09-14T07:11:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156348+0000", 2024-09-14T07:11:52.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8499999999999998e-06 2024-09-14T07:11:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156360+0000", 2024-09-14T07:11:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2367e-05 2024-09-14T07:11:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156392+0000", 2024-09-14T07:11:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2449000000000001e-05 2024-09-14T07:11:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156410+0000", 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.753e-05 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156537+0000", 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012719900000000001 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156555+0000", 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7887e-05 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103715+0000", 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.94715964900000005 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.103774+0000", 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9658999999999999e-05 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.281 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4354 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 202.10520676799999, 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95163299499999998, 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4354 2024-09-14T07:11:52.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156447+0000", 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156448+0000", 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.156e-06 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156453+0000", 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156465+0000", 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2533e-05 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156476+0000", 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0913e-05 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156639+0000", 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162893 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.156665+0000", 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5842000000000001e-05 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108036+0000", 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.95137164200000002 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108080+0000", 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708999999999998e-05 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.286 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.287 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4449.0:4355 3.9 3.41433489 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 201.40615492699999, 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25280286499999999, 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4449", 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 4355 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855499+0000", 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855502+0000", 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9129999999999999e-06 2024-09-14T07:11:52.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855512+0000", 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0523000000000001e-05 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855545+0000", 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2866e-05 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855563+0000", 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855656+0000", 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3168e-05 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:30.855667+0000", 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0886e-05 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108280+0000", 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.25261326000000001 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:31.108302+0000", 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1866e-05 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.292 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1706 3.1a 3:5d7f6e15:::rbd_data.116150b87d44.0000000000000005:14 [read 649216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.921143366, 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093429017000000003, 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1706 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340510+0000", 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340511+0000", 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3300000000000001e-07 2024-09-14T07:11:52.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340514+0000", 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1769999999999998e-06 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340526+0000", 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1963e-05 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340534+0000", 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.576e-06 2024-09-14T07:11:52.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433509+0000", 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092975414000000006 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433583+0000", 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3332999999999997e-05 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433939+0000", 2024-09-14T07:11:52.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000356721 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1707 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.92110929699999, 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440112000000006, 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1707 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340545+0000", 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8700000000000005e-07 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340548+0000", 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.926e-06 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340557+0000", 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8230000000000001e-06 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340576+0000", 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.948e-05 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433976+0000", 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093399472999999997 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.433985+0000", 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7229999999999993e-06 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1708 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.921065951, 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093433453, 2024-09-14T07:11:52.301 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1708 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340588+0000", 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340589+0000", 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8400000000000003e-07 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340592+0000", 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2629999999999999e-06 2024-09-14T07:11:52.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340604+0000", 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1953000000000001e-05 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340611+0000", 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1470000000000004e-06 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434014+0000", 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093403018000000004 2024-09-14T07:11:52.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434021+0000", 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1709 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.92103242799999, 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093440299000000004, 2024-09-14T07:11:52.305 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1709 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340621+0000", 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340622+0000", 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.23e-07 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340625+0000", 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.847e-06 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340637+0000", 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1562999999999999e-05 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340648+0000", 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1060000000000001e-05 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434055+0000", 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093407723999999998 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434062+0000", 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3820000000000001e-06 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1710 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.920985178, 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093414769999999994, 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.309 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1710 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340669+0000", 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340670+0000", 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.096e-06 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340686+0000", 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.609e-05 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340708+0000", 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2413e-05 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340758+0000", 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9709000000000001e-05 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434077+0000", 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318841999999999 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434083+0000", 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6200000000000001e-06 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1711 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.92087093000001, 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093318330000000005, 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.313 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1711 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340783+0000", 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340784+0000", 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.28e-06 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340791+0000", 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4400000000000002e-06 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340850+0000", 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9565000000000003e-05 2024-09-14T07:11:52.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340868+0000", 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7574000000000001e-05 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434098+0000", 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093229804999999999 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434101+0000", 2024-09-14T07:11:52.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1712 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.92076123499999, 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093225807999999993, 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.317 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1712 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340893+0000", 2024-09-14T07:11:52.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340894+0000", 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340901+0000", 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4099999999999996e-06 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340924+0000", 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3053000000000001e-05 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340939+0000", 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5053e-05 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434115+0000", 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093175959000000003 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434118+0000", 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8229999999999997e-06 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1713 3.1a 3.a876feba (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 165.92069509199999, 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093178603999999998, 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.321 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "reached pg", 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1713 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340959+0000", 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340961+0000", 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8029999999999999e-06 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.340979+0000", 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8335999999999999e-05 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341020+0000", 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1425999999999999e-05 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.341037+0000", 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6847e-05 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434133+0000", 2024-09-14T07:11:52.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093096250000000005 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.434137+0000", 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5713.0:1143 3.b 3:d19810a4:::rbd_data.11618d2dcb2b.0000000000000002:head [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 109.68561924399999, 2024-09-14T07:11:52.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.19989332100000001, 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5713", 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/4161014365", 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1143 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:52.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576035+0000", 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576036+0000", 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.528e-06 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576041+0000", 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0549999999999999e-06 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576058+0000", 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241e-05 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576068+0000", 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5019999999999998e-06 2024-09-14T07:11:52.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576132+0000", 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4329e-05 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.576141+0000", 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1959999999999997e-06 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775510+0000", 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.199368145 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775717+0000", 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020770000000000001 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:02.775928+0000", 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021062500000000001 2024-09-14T07:11:52.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.6120.0:197 3.0 3:032d7705:::rbd_object_map.1161eb6123dc:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 100.39890580799999, 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96521883900000005, 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.331 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.6120", 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/55585913", 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 197 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862748+0000", 2024-09-14T07:11:52.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862750+0000", 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9400000000000001e-06 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862758+0000", 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3799999999999994e-06 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862778+0000", 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9412999999999999e-05 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862789+0000", 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0807e-05 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862896+0000", 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107388 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:11.862915+0000", 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9530000000000001e-05 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825797+0000", 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.96288185000000004 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.825951+0000", 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015359400000000001 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826156+0000", 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000205043 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.826960+0000", 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080353800000000002 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827889+0000", 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000929053 2024-09-14T07:11:52.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827933+0000", 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4400999999999997e-05 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:12.827967+0000", 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3902000000000002e-05 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:849 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.782890765999994, 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093383062000000003, 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 849 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478763+0000", 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478764+0000", 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1599999999999999e-06 2024-09-14T07:11:52.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478772+0000", 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3540000000000002e-06 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478803+0000", 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0985999999999998e-05 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478814+0000", 2024-09-14T07:11:52.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1173e-05 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567774+0000", 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088959776000000004 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.567866+0000", 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.268e-05 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568184+0000", 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031811200000000001 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569941+0000", 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017562669999999999 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572046+0000", 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002105034 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572103+0000", 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7581999999999999e-05 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572146+0000", 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2938000000000001e-05 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:850 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.782827066999999, 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093485294999999996, 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 850 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478827+0000", 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478828+0000", 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3399999999999998e-07 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478832+0000", 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478842+0000", 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6469999999999993e-06 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478852+0000", 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0285999999999999e-05 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568638+0000", 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089785798 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.568739+0000", 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101134 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569157+0000", 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417892 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572161+0000", 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030046740000000001 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572214+0000", 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2528999999999998e-05 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572272+0000", 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8137999999999998e-05 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572312+0000", 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.998e-05 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:851 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.782783733000002, 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094323610000000002, 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.349 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 851 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478870+0000", 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478871+0000", 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.26e-06 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478878+0000", 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1330000000000001e-06 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478897+0000", 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8096000000000001e-05 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478909+0000", 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2813000000000001e-05 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.569992+0000", 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091082179999999999 2024-09-14T07:11:52.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570080+0000", 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8585000000000007e-05 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.570515+0000", 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043522800000000001 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573033+0000", 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025179769999999998 2024-09-14T07:11:52.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573074+0000", 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0195999999999999e-05 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573130+0000", 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5945000000000001e-05 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573194+0000", 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4196999999999995e-05 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.354 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:852 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.782726854000003, 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094757626999999997, 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:52.355 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 852 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478927+0000", 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478928+0000", 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478935+0000", 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8940000000000004e-06 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478953+0000", 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7822999999999999e-05 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478967+0000", 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.382e-05 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571108+0000", 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.092141739 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571185+0000", 2024-09-14T07:11:52.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6939000000000003e-05 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.571508+0000", 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032255400000000002 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573212+0000", 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017041700000000001 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573588+0000", 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00037626800000000002 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573643+0000", 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4456999999999997e-05 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573685+0000", 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1727000000000002e-05 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.360 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:853 3.19 3:9b3789d5:::rbd_object_map.1161264f5ad6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.782660359000005, 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096730712999999996, 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:52.361 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 853 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478993+0000", 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.478995+0000", 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2899999999999999e-06 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479002+0000", 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6140000000000003e-06 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479025+0000", 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2733000000000001e-05 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.479041+0000", 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6027000000000001e-05 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572364+0000", 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.093323097999999993 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572430+0000", 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5605999999999995e-05 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.572725+0000", 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029492800000000002 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.574660+0000", 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019348810000000001 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575628+0000", 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00096860499999999999 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575680+0000", 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1345999999999999e-05 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.575724+0000", 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4585000000000001e-05 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.367 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5681.0:859 3.9 3:93476647:::rbd_data.1161264f5ad6.00000000000002a0:head [set-alloc-hint object_size 262144 write_size 262144,write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134)", 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 96.779776021999993, 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.095076874000000006, 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5681", 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1525253623", 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 859 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.368 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481878+0000", 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481880+0000", 2024-09-14T07:11:52.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.182e-06 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481906+0000", 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6293999999999999e-05 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481927+0000", 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0497e-05 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.481940+0000", 2024-09-14T07:11:52.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2757e-05 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573245+0000", 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.091305788999999998 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573381+0000", 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013560800000000001 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.573472+0000", 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1272000000000003e-05 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576513+0000", 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030406249999999999 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576867+0000", 2024-09-14T07:11:52.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035405799999999998 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576918+0000", 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0911000000000001e-05 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:15.576955+0000", 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6881000000000001e-05 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.373 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.374 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.374 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.374 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:52.376 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:11:52.377 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:52.378 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-14T07:11:51.829823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128717872,"num_objects":345,"num_object_clones":10,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":255,"num_read":340638,"num_read_kb":1115632,"num_write":21938,"num_write_kb":331477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19287,"ondisk_log_size":19287,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":350588,"kb_used_data":18336,"kb_used_omap":35,"kb_used_meta":331996,"kb_avail":565880452,"statfs":{"total":579820584960,"available":579461582848,"internally_reserved":0,"allocated":18776064,"data_stored":26346232,"data_compressed":1453205,"data_compressed_allocated":2420736,"data_compressed_original":19505152,"omap_allocated":36329,"internal_metadata":339964439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-134003597,"num_objects":-98,"num_object_clones":-18,"num_object_copies":-196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-98,"num_whiteouts":-15,"num_read":549,"num_read_kb":467,"num_write":143,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":398,"num_bytes_recovered":1123856,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.045557"},"pg_stats":[{"pgid":"3.1f","version":"217'381","reported_seq":6950,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820211+0000","last_change":"2024-09-14T07:11:49.823797+0000","last_active":"2024-09-14T07:11:50.820211+0000","last_peered":"2024-09-14T07:11:50.820211+0000","last_clean":"2024-09-14T07:11:50.820211+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:11:50.820211+0000","last_undegraded":"2024-09-14T07:11:50.820211+0000","last_fullsized":"2024-09-14T07:11:50.820211+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:04:11.679482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081229,"stat_sum":{"num_bytes":16,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6401,"num_read_kb":50527,"num_write":555,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1e","version":"217'720","reported_seq":12973,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001526+0000","last_change":"2024-09-14T07:11:49.832703+0000","last_active":"2024-09-14T07:11:50.001526+0000","last_peered":"2024-09-14T07:11:50.001526+0000","last_clean":"2024-09-14T07:11:50.001526+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:11:50.001526+0000","last_undegraded":"2024-09-14T07:11:50.001526+0000","last_fullsized":"2024-09-14T07:11:50.001526+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:57:12.557671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077816990000000004,"stat_sum":{"num_bytes":5972628,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":14175,"num_read_kb":59693,"num_write":943,"num_write_kb":21289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1d","version":"217'1602","reported_seq":16581,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005381+0000","last_change":"2024-09-14T07:11:49.834031+0000","last_active":"2024-09-14T07:11:50.005381+0000","last_peered":"2024-09-14T07:11:50.005381+0000","last_clean":"2024-09-14T07:11:50.005381+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:11:50.005381+0000","last_undegraded":"2024-09-14T07:11:50.005381+0000","last_fullsized":"2024-09-14T07:11:50.005381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1602,"log_dups_size":0,"ondisk_log_size":1602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:22:12.091166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072780320000000003,"stat_sum":{"num_bytes":4808704,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":20893,"num_read_kb":36306,"num_write":1696,"num_write_kb":5676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1c","version":"217'501","reported_seq":20791,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.895045+0000","last_change":"2024-09-14T07:11:49.826028+0000","last_active":"2024-09-14T07:11:49.895045+0000","last_peered":"2024-09-14T07:11:49.895045+0000","last_clean":"2024-09-14T07:11:49.895045+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:11:49.895045+0000","last_undegraded":"2024-09-14T07:11:49.895045+0000","last_fullsized":"2024-09-14T07:11:49.895045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:25.506830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001223399,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21030,"num_read_kb":46630,"num_write":501,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1b","version":"214'572","reported_seq":8666,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819776+0000","last_change":"2024-09-14T07:11:49.823068+0000","last_active":"2024-09-14T07:11:50.819776+0000","last_peered":"2024-09-14T07:11:50.819776+0000","last_clean":"2024-09-14T07:11:50.819776+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:11:50.819776+0000","last_undegraded":"2024-09-14T07:11:50.819776+0000","last_fullsized":"2024-09-14T07:11:50.819776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:06:03.189849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015995510000000001,"stat_sum":{"num_bytes":19174036,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":8775,"num_read_kb":22847,"num_write":713,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1a","version":"213'389","reported_seq":30287,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.884017+0000","last_change":"2024-09-14T07:11:46.814896+0000","last_active":"2024-09-14T07:11:46.884017+0000","last_peered":"2024-09-14T07:11:46.884017+0000","last_clean":"2024-09-14T07:11:46.884017+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:46.884017+0000","last_undegraded":"2024-09-14T07:11:46.884017+0000","last_fullsized":"2024-09-14T07:11:46.884017+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:22.158079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010132499999999999,"stat_sum":{"num_bytes":786451,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":6,"num_read":29642,"num_read_kb":50524,"num_write":413,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"3.19","version":"217'1338","reported_seq":10587,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822350+0000","last_change":"2024-09-14T07:11:49.822350+0000","last_active":"2024-09-14T07:11:49.822350+0000","last_peered":"2024-09-14T07:11:49.822350+0000","last_clean":"2024-09-14T07:11:49.822350+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:11:49.822350+0000","last_undegraded":"2024-09-14T07:11:49.822350+0000","last_fullsized":"2024-09-14T07:11:49.822350+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1338,"log_dups_size":0,"ondisk_log_size":1338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:39.514580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042770159999999998,"stat_sum":{"num_bytes":2836518,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":14561,"num_read_kb":35933,"num_write":2117,"num_write_kb":9607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.18","version":"218'427","reported_seq":5411,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.949888+0000","last_change":"2024-09-14T07:11:49.822391+0000","last_active":"2024-09-14T07:11:50.949888+0000","last_peered":"2024-09-14T07:11:50.949888+0000","last_clean":"2024-09-14T07:11:50.949888+0000","last_became_active":"2024-09-14T07:11:42.792094+0000","last_became_peered":"2024-09-14T07:11:42.792094+0000","last_unstale":"2024-09-14T07:11:50.949888+0000","last_undegraded":"2024-09-14T07:11:50.949888+0000","last_fullsized":"2024-09-14T07:11:50.949888+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:42:01.973449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096171600000000005,"stat_sum":{"num_bytes":270384,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":4861,"num_read_kb":15257,"num_write":485,"num_write_kb":15533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.17","version":"217'242","reported_seq":3834,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005660+0000","last_change":"2024-09-14T07:11:49.834634+0000","last_active":"2024-09-14T07:11:50.005660+0000","last_peered":"2024-09-14T07:11:50.005660+0000","last_clean":"2024-09-14T07:11:50.005660+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:11:50.005660+0000","last_undegraded":"2024-09-14T07:11:50.005660+0000","last_fullsized":"2024-09-14T07:11:50.005660+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:55.293205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.16","version":"216'640","reported_seq":9998,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.573145+0000","last_change":"2024-09-14T07:11:50.307883+0000","last_active":"2024-09-14T07:11:50.307883+0000","last_peered":"2024-09-14T07:11:49.573145+0000","last_clean":"2024-09-14T07:11:43.788063+0000","last_became_active":"2024-09-14T07:11:48.815066+0000","last_became_peered":"2024-09-14T07:11:48.815066+0000","last_unstale":"2024-09-14T07:11:49.573145+0000","last_undegraded":"2024-09-14T07:11:49.573145+0000","last_fullsized":"2024-09-14T07:11:49.573145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084910960000000001,"stat_sum":{"num_bytes":1368064,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":738,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.15","version":"217'509","reported_seq":6975,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.833831+0000","last_change":"2024-09-14T07:11:49.833831+0000","last_active":"2024-09-14T07:11:49.833831+0000","last_peered":"2024-09-14T07:11:49.833831+0000","last_clean":"2024-09-14T07:11:49.833831+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:11:49.833831+0000","last_undegraded":"2024-09-14T07:11:49.833831+0000","last_fullsized":"2024-09-14T07:11:49.833831+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:17.752405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075586000000000004,"stat_sum":{"num_bytes":4245041,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":7401,"num_read_kb":47217,"num_write":522,"num_write_kb":16662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.14","version":"217'750","reported_seq":11076,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.933900+0000","last_change":"2024-09-14T07:11:49.830868+0000","last_active":"2024-09-14T07:11:49.933900+0000","last_peered":"2024-09-14T07:11:49.933900+0000","last_clean":"2024-09-14T07:11:49.933900+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:11:49.933900+0000","last_undegraded":"2024-09-14T07:11:49.933900+0000","last_fullsized":"2024-09-14T07:11:49.933900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:21.610420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066874480000000004,"stat_sum":{"num_bytes":12418560,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":12254,"num_read_kb":39191,"num_write":924,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.7","version":"0'0","reported_seq":429,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826029+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:11:49.826029+0000","last_peered":"2024-09-14T07:11:49.826029+0000","last_clean":"2024-09-14T07:11:49.826029+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:11:49.826029+0000","last_undegraded":"2024-09-14T07:11:49.826029+0000","last_fullsized":"2024-09-14T07:11:49.826029+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"218'692","reported_seq":12786,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820180+0000","last_change":"2024-09-14T07:11:49.823661+0000","last_active":"2024-09-14T07:11:50.820180+0000","last_peered":"2024-09-14T07:11:50.820180+0000","last_clean":"2024-09-14T07:11:50.820180+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:11:50.820180+0000","last_undegraded":"2024-09-14T07:11:50.820180+0000","last_fullsized":"2024-09-14T07:11:50.820180+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:52:36.850781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001773719,"stat_sum":{"num_bytes":129628,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":14012,"num_read_kb":35141,"num_write":738,"num_write_kb":5113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.819091+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:11:49.819091+0000","last_peered":"2024-09-14T07:11:49.819091+0000","last_clean":"2024-09-14T07:11:49.819091+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:11:49.819091+0000","last_undegraded":"2024-09-14T07:11:49.819091+0000","last_fullsized":"2024-09-14T07:11:49.819091+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"217'658","reported_seq":8317,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001581+0000","last_change":"2024-09-14T07:11:49.820591+0000","last_active":"2024-09-14T07:11:50.001581+0000","last_peered":"2024-09-14T07:11:50.001581+0000","last_clean":"2024-09-14T07:11:50.001581+0000","last_became_active":"2024-09-14T07:10:57.225048+0000","last_became_peered":"2024-09-14T07:10:57.225048+0000","last_unstale":"2024-09-14T07:11:50.001581+0000","last_undegraded":"2024-09-14T07:11:50.001581+0000","last_fullsized":"2024-09-14T07:11:50.001581+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:30:34.825320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025417880000000001,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":701,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.5","version":"0'0","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.827578+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:11:49.827578+0000","last_peered":"2024-09-14T07:11:49.827578+0000","last_clean":"2024-09-14T07:11:49.827578+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:11:49.827578+0000","last_undegraded":"2024-09-14T07:11:49.827578+0000","last_fullsized":"2024-09-14T07:11:49.827578+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"216'686","reported_seq":6671,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.821973+0000","last_change":"2024-09-14T07:11:49.822694+0000","last_active":"2024-09-14T07:11:49.821973+0000","last_peered":"2024-09-14T07:11:49.821973+0000","last_clean":"2024-09-14T07:11:49.821973+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:11:49.821973+0000","last_undegraded":"2024-09-14T07:11:49.821973+0000","last_fullsized":"2024-09-14T07:11:49.821973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:34:36.123624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042077470000000004,"stat_sum":{"num_bytes":131072,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":862,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.818494+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:11:49.818494+0000","last_peered":"2024-09-14T07:11:49.818494+0000","last_clean":"2024-09-14T07:11:49.818494+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:11:49.818494+0000","last_undegraded":"2024-09-14T07:11:49.818494+0000","last_fullsized":"2024-09-14T07:11:49.818494+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"217'750","reported_seq":4024,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.994308+0000","last_change":"2024-09-14T07:11:49.834054+0000","last_active":"2024-09-14T07:11:49.994308+0000","last_peered":"2024-09-14T07:11:49.994308+0000","last_clean":"2024-09-14T07:11:49.994308+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:11:49.994308+0000","last_undegraded":"2024-09-14T07:11:49.994308+0000","last_fullsized":"2024-09-14T07:11:49.994308+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:44:07.647554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006901735,"stat_sum":{"num_bytes":5210128,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":5230,"num_read_kb":19720,"num_write":1110,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.3","version":"214'489","reported_seq":10033,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.927980+0000","last_change":"2024-09-14T07:11:46.810220+0000","last_active":"2024-09-14T07:11:46.927980+0000","last_peered":"2024-09-14T07:11:46.927980+0000","last_clean":"2024-09-14T07:11:46.927980+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:11:46.927980+0000","last_undegraded":"2024-09-14T07:11:46.927980+0000","last_fullsized":"2024-09-14T07:11:46.927980+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:40:26.998349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055825800000000002,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8723,"num_read_kb":48182,"num_write":528,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"2.2","version":"19'2","reported_seq":440,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826271+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:11:49.826271+0000","last_peered":"2024-09-14T07:11:49.826271+0000","last_clean":"2024-09-14T07:11:49.826271+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:11:49.826271+0000","last_undegraded":"2024-09-14T07:11:49.826271+0000","last_fullsized":"2024-09-14T07:11:49.826271+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"217'500","reported_seq":6349,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993429+0000","last_change":"2024-09-14T07:11:49.823563+0000","last_active":"2024-09-14T07:11:49.993429+0000","last_peered":"2024-09-14T07:11:49.993429+0000","last_clean":"2024-09-14T07:11:49.993429+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:11:49.993429+0000","last_undegraded":"2024-09-14T07:11:49.993429+0000","last_fullsized":"2024-09-14T07:11:49.993429+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:33.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042663839999999998,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7616,"num_read_kb":23803,"num_write":401,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.817757+0000","last_change":"2024-09-14T07:10:52.793359+0000","last_active":"2024-09-14T07:11:49.817757+0000","last_peered":"2024-09-14T07:11:49.817757+0000","last_clean":"2024-09-14T07:11:49.817757+0000","last_became_active":"2024-09-14T07:10:52.792887+0000","last_became_peered":"2024-09-14T07:10:52.792887+0000","last_unstale":"2024-09-14T07:11:49.817757+0000","last_undegraded":"2024-09-14T07:11:49.817757+0000","last_fullsized":"2024-09-14T07:11:49.817757+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:11.029506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":12984,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819542+0000","last_change":"2024-09-14T07:11:49.822587+0000","last_active":"2024-09-14T07:11:50.819542+0000","last_peered":"2024-09-14T07:11:50.819542+0000","last_clean":"2024-09-14T07:11:50.819542+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:11:50.819542+0000","last_undegraded":"2024-09-14T07:11:50.819542+0000","last_fullsized":"2024-09-14T07:11:50.819542+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:12:01.776651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001171233,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.0","version":"0'0","reported_seq":430,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822692+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:11:49.822692+0000","last_peered":"2024-09-14T07:11:49.822692+0000","last_clean":"2024-09-14T07:11:49.822692+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:11:49.822692+0000","last_undegraded":"2024-09-14T07:11:49.822692+0000","last_fullsized":"2024-09-14T07:11:49.822692+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"213'396","reported_seq":7828,"reported_epoch":216,"state":"peering","last_fresh":"2024-09-14T07:11:48.983941+0000","last_change":"2024-09-14T07:11:49.818862+0000","last_active":"2024-09-14T07:11:49.818862+0000","last_peered":"2024-09-14T07:11:48.983941+0000","last_clean":"2024-09-14T07:11:46.103525+0000","last_became_active":"2024-09-14T07:11:48.811974+0000","last_became_peered":"2024-09-14T07:11:48.811974+0000","last_unstale":"2024-09-14T07:11:48.983941+0000","last_undegraded":"2024-09-14T07:11:48.983941+0000","last_fullsized":"2024-09-14T07:11:48.794907+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01901893,"stat_sum":{"num_bytes":858640,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":6436,"num_read_kb":16010,"num_write":342,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"2.3","version":"17'1","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.825168+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:11:49.825168+0000","last_peered":"2024-09-14T07:11:49.825168+0000","last_clean":"2024-09-14T07:11:49.825168+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:11:49.825168+0000","last_undegraded":"2024-09-14T07:11:49.825168+0000","last_fullsized":"2024-09-14T07:11:49.825168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":542,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822949+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:11:49.822949+0000","last_peered":"2024-09-14T07:11:49.822949+0000","last_clean":"2024-09-14T07:11:49.822949+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:11:49.822949+0000","last_undegraded":"2024-09-14T07:11:49.822949+0000","last_fullsized":"2024-09-14T07:11:49.822949+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"217'535","reported_seq":10138,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.820793+0000","last_change":"2024-09-14T07:11:49.820793+0000","last_active":"2024-09-14T07:11:49.820793+0000","last_peered":"2024-09-14T07:11:49.820793+0000","last_clean":"2024-09-14T07:11:49.820793+0000","last_became_active":"2024-09-14T07:10:58.788140+0000","last_became_peered":"2024-09-14T07:10:58.788140+0000","last_unstale":"2024-09-14T07:11:49.820793+0000","last_undegraded":"2024-09-14T07:11:49.820793+0000","last_fullsized":"2024-09-14T07:11:49.820793+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:21:00.558281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002853487,"stat_sum":{"num_bytes":7600655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":9538,"num_read_kb":69033,"num_write":685,"num_write_kb":27473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.9","version":"214'740","reported_seq":8333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-14T07:11:46.824036+0000","last_change":"2024-09-14T07:11:46.827725+0000","last_active":"2024-09-14T07:11:46.824036+0000","last_peered":"2024-09-14T07:11:46.824036+0000","last_clean":"2024-09-14T07:11:46.824036+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:11:46.824036+0000","last_undegraded":"2024-09-14T07:11:46.824036+0000","last_fullsized":"2024-09-14T07:11:46.824036+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:27:43.003401+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.021946032000000001,"stat_sum":{"num_bytes":568040,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":10,"num_read":10159,"num_read_kb":48108,"num_write":750,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"9","length":"1b"}]},{"pgid":"3.a","version":"217'437","reported_seq":8196,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.835504+0000","last_change":"2024-09-14T07:11:49.835504+0000","last_active":"2024-09-14T07:11:49.835504+0000","last_peered":"2024-09-14T07:11:49.835504+0000","last_clean":"2024-09-14T07:11:49.835504+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:11:49.835504+0000","last_undegraded":"2024-09-14T07:11:49.835504+0000","last_fullsized":"2024-09-14T07:11:49.835504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:24.697088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047041990000000001,"stat_sum":{"num_bytes":1452032,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":7826,"num_read_kb":35943,"num_write":413,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.b","version":"217'1019","reported_seq":8107,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.985806+0000","last_change":"2024-09-14T07:11:49.821839+0000","last_active":"2024-09-14T07:11:49.985806+0000","last_peered":"2024-09-14T07:11:49.985806+0000","last_clean":"2024-09-14T07:11:49.985806+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:11:49.985806+0000","last_undegraded":"2024-09-14T07:11:49.985806+0000","last_fullsized":"2024-09-14T07:11:49.985806+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:20:17.413757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036922539999999998,"stat_sum":{"num_bytes":393384,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":10228,"num_read_kb":41579,"num_write":1377,"num_write_kb":13967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.c","version":"217'547","reported_seq":15456,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830180+0000","last_change":"2024-09-14T07:11:49.830180+0000","last_active":"2024-09-14T07:11:49.830180+0000","last_peered":"2024-09-14T07:11:49.830180+0000","last_clean":"2024-09-14T07:11:49.830180+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:49.830180+0000","last_undegraded":"2024-09-14T07:11:49.830180+0000","last_fullsized":"2024-09-14T07:11:49.830180+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:03.498897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039997849999999996,"stat_sum":{"num_bytes":262144,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":463,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.d","version":"217'342","reported_seq":4125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993312+0000","last_change":"2024-09-14T07:11:49.834171+0000","last_active":"2024-09-14T07:11:49.993312+0000","last_peered":"2024-09-14T07:11:49.993312+0000","last_clean":"2024-09-14T07:11:49.993312+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:11:49.993312+0000","last_undegraded":"2024-09-14T07:11:49.993312+0000","last_fullsized":"2024-09-14T07:11:49.993312+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:13.447198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071541449999999998,"stat_sum":{"num_bytes":1048576,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":323,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.e","version":"216'443","reported_seq":17414,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.304818+0000","last_change":"2024-09-14T07:11:50.307914+0000","last_active":"2024-09-14T07:11:50.307914+0000","last_peered":"2024-09-14T07:11:49.304818+0000","last_clean":"2024-09-14T07:11:40.083699+0000","last_became_active":"2024-09-14T07:11:48.814324+0000","last_became_peered":"2024-09-14T07:11:48.814324+0000","last_unstale":"2024-09-14T07:11:49.304818+0000","last_undegraded":"2024-09-14T07:11:49.304818+0000","last_fullsized":"2024-09-14T07:11:49.304818+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0016376439999999999,"stat_sum":{"num_bytes":524288,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":438,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.f","version":"217'484","reported_seq":8518,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830984+0000","last_change":"2024-09-14T07:11:49.830984+0000","last_active":"2024-09-14T07:11:49.830984+0000","last_peered":"2024-09-14T07:11:49.830984+0000","last_clean":"2024-09-14T07:11:49.830984+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:11:49.830984+0000","last_undegraded":"2024-09-14T07:11:49.830984+0000","last_fullsized":"2024-09-14T07:11:49.830984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:49:26.120361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004728945,"stat_sum":{"num_bytes":131072,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":615,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.10","version":"216'462","reported_seq":8117,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.138923+0000","last_change":"2024-09-14T07:11:50.307757+0000","last_active":"2024-09-14T07:11:50.307757+0000","last_peered":"2024-09-14T07:11:49.138923+0000","last_clean":"2024-09-14T07:11:45.797283+0000","last_became_active":"2024-09-14T07:11:48.810645+0000","last_became_peered":"2024-09-14T07:11:48.810645+0000","last_unstale":"2024-09-14T07:11:49.138923+0000","last_undegraded":"2024-09-14T07:11:49.138923+0000","last_fullsized":"2024-09-14T07:11:48.791074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023065630000000002,"stat_sum":{"num_bytes":7864320,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":651,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.11","version":"217'537","reported_seq":9421,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826638+0000","last_change":"2024-09-14T07:11:49.826638+0000","last_active":"2024-09-14T07:11:49.826638+0000","last_peered":"2024-09-14T07:11:49.826638+0000","last_clean":"2024-09-14T07:11:49.826638+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:11:49.826638+0000","last_undegraded":"2024-09-14T07:11:49.826638+0000","last_fullsized":"2024-09-14T07:11:49.826638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:22:42.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036882769999999998,"stat_sum":{"num_bytes":17920512,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":10491,"num_read_kb":24428,"num_write":419,"num_write_kb":9181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.12","version":"217'558","reported_seq":7288,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.997627+0000","last_change":"2024-09-14T07:11:49.835043+0000","last_active":"2024-09-14T07:11:49.997627+0000","last_peered":"2024-09-14T07:11:49.997627+0000","last_clean":"2024-09-14T07:11:49.997627+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:11:49.997627+0000","last_undegraded":"2024-09-14T07:11:49.997627+0000","last_fullsized":"2024-09-14T07:11:49.997627+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:25:26.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005177877,"stat_sum":{"num_bytes":18746368,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":428,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.13","version":"217'381","reported_seq":4946,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.831753+0000","last_change":"2024-09-14T07:11:49.831753+0000","last_active":"2024-09-14T07:11:49.831753+0000","last_peered":"2024-09-14T07:11:49.831753+0000","last_clean":"2024-09-14T07:11:49.831753+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:11:49.831753+0000","last_undegraded":"2024-09-14T07:11:49.831753+0000","last_fullsized":"2024-09-14T07:11:49.831753+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:29.134495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073722479999999996,"stat_sum":{"num_bytes":9211904,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":356,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":128127485,"num_objects":341,"num_object_clones":10,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":255,"num_read":340572,"num_read_kb":1115577,"num_write":21835,"num_write_kb":329847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2539520,"data_stored":13576088,"data_compressed":1362915,"data_compressed_allocated":1904640,"data_compressed_original":13279232,"omap_allocated":12989,"internal_metadata":0},"log_size":19212,"ondisk_log_size":19212,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78684,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293156,"statfs":{"total":96636764160,"available":96556191744,"internally_reserved":0,"allocated":3145728,"data_stored":3831785,"data_compressed":256797,"data_compressed_allocated":425984,"data_compressed_original":2682880,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607608,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52124,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319716,"statfs":{"total":96636764160,"available":96583389184,"internally_reserved":0,"allocated":2957312,"data_stored":3660859,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607608,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70536,"kb_used_data":3196,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301304,"statfs":{"total":96636764160,"available":96564535296,"internally_reserved":0,"allocated":3272704,"data_stored":4258061,"data_compressed":351328,"data_compressed_allocated":512000,"data_compressed_original":3088384,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607609,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64988,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306852,"statfs":{"total":96636764160,"available":96570216448,"internally_reserved":0,"allocated":3330048,"data_stored":6111343,"data_compressed":270417,"data_compressed_allocated":561152,"data_compressed_original":4960256,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607609,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48052,"kb_used_data":2980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323788,"statfs":{"total":96636764160,"available":96587558912,"internally_reserved":0,"allocated":3051520,"data_stored":4216320,"data_compressed":171212,"data_compressed_allocated":348160,"data_compressed_original":3092480,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":2948,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":3018752,"data_stored":4267864,"data_compressed":256234,"data_compressed_allocated":327680,"data_compressed_original":3162112,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":2587034,"data_compressed":249342,"data_compressed_allocated":290816,"data_compressed_original":2572288,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2034009,"data_compressed":161836,"data_compressed_allocated":258048,"data_compressed_original":2002944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":1396187,"data_compressed":231909,"data_compressed_allocated":282624,"data_compressed_original":1343488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":2582245,"data_compressed":345270,"data_compressed_allocated":475136,"data_compressed_original":2514944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":487424,"data_stored":2401221,"data_compressed":227341,"data_compressed_allocated":352256,"data_compressed_original":2326528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2575392,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:11:52.380 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:52.373+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:52.476 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:52.473+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:52.546 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:52.547 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:52.547 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-14T07:11:52.547 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:52.561 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:52.608 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:52.605+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:52.678 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:52.673+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:52.679 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-14T07:11:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-14T07:11:52.679 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-14T07:11:52.679 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:52.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:52.778 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:52.773+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:52.864 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:924 3.5 3:a3be529b:::rbd_data.11611b5ca054.0000000000000000:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 240.03143819300001, 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006503682, 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 924 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830537+0000", 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830538+0000", 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.226e-06 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830543+0000", 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830559+0000", 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5089000000000001e-05 2024-09-14T07:11:52.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830567+0000", 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4849999999999992e-06 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830677+0000", 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000109899 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.830702+0000", 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4559999999999999e-05 2024-09-14T07:11:52.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836737+0000", 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.006035918 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.836898+0000", 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016057300000000001 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837041+0000", 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014274399999999999 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4406.0:925 3.5 3:a779e04e:::rbd_data.11611b5ca054.0000000000000002:head [read 978944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 240.03066384499999, 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0060956029999999, 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4406", 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/140933223", 2024-09-14T07:11:52.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 925 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831311+0000", 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831312+0000", 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.06e-06 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831317+0000", 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831331+0000", 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3728e-05 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831347+0000", 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6257e-05 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831417+0000", 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9831000000000003e-05 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:52.831428+0000", 2024-09-14T07:11:52.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1311999999999999e-05 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837171+0000", 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.00574349 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837275+0000", 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104094 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:07:53.837407+0000", 2024-09-14T07:11:52.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000131536 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:530 3.f 3:f4caec87:::rbd_data.1161ff1af5c4.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 19545600~29184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 211.01948084700001, 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.096139219999999997, 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 530 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842494+0000", 2024-09-14T07:11:52.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.842497+0000", 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2060000000000001e-06 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858663+0000", 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.016166656000000001 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858696+0000", 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2289000000000002e-05 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858708+0000", 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2687000000000001e-05 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858794+0000", 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5290999999999994e-05 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.858952+0000", 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000158732 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.860568+0000", 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001616103 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.927924+0000", 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.067355633999999998 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938463+0000", 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010538607 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938550+0000", 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7536999999999999e-05 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:21.938634+0000", 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3478000000000003e-05 2024-09-14T07:11:52.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4703.0:831 3.10 3:0b70df4e:::rbd_data.1161ff1af5c4.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 32268288~52224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 210.72255264099999, 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.111563996, 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4703", 2024-09-14T07:11:52.881 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2563731855", 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 831 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139423+0000", 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.139426+0000", 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151515+0000", 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012089259999999999 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151559+0000", 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3698000000000001e-05 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151578+0000", 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9363000000000001e-05 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151721+0000", 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000142855 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.151969+0000", 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024778000000000002 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-14T07:11:52.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.155287+0000", 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003318325 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250223+0000", 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.094935828 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250694+0000", 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00047082100000000001 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250879+0000", 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018518199999999999 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:08:22.250987+0000", 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107898 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1873 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 481280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.48764881700001, 2024-09-14T07:11:52.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088001961000000004, 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1873 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374326+0000", 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374795+0000", 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00046841999999999999 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374803+0000", 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374836+0000", 2024-09-14T07:11:52.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3389000000000002e-05 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374861+0000", 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5170000000000001e-05 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.438797+0000", 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.063935793000000005 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.461599+0000", 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.022801859000000001 2024-09-14T07:11:52.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462328+0000", 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00072940400000000001 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1874 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.48708702600001, 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087624899000000006, 2024-09-14T07:11:52.890 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1874 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.374888+0000", 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375305+0000", 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000417076 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375312+0000", 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.092e-06 2024-09-14T07:11:52.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375332+0000", 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9395e-05 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375360+0000", 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8051999999999998e-05 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462363+0000", 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087003286999999999 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462433+0000", 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462513+0000", 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9826000000000004e-05 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.894 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1875 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 526336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.48659245900001, 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.087442955000000003, 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1875 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375383+0000", 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375887+0000", 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050465899999999997 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375921+0000", 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3649000000000003e-05 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375949+0000", 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7501999999999999e-05 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.375974+0000", 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5384e-05 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462541+0000", 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086566650999999994 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462746+0000", 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020551 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462826+0000", 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9599999999999997e-05 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.898 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1876 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 548864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.48597504, 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086994327999999996, 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1876 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376000+0000", 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376379+0000", 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000378305 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376385+0000", 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9280000000000004e-06 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376401+0000", 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5957e-05 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376411+0000", 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0929999999999992e-06 2024-09-14T07:11:52.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462854+0000", 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086443359999999997 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462920+0000", 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5745000000000002e-05 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.462995+0000", 2024-09-14T07:11:52.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1877 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 571392~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.48554678100001, 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086746261000000005, 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.903 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1877 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376428+0000", 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376765+0000", 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033674099999999999 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376773+0000", 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6899999999999992e-06 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376790+0000", 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7241999999999999e-05 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376802+0000", 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2264e-05 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463020+0000", 2024-09-14T07:11:52.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086217481999999998 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463086+0000", 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6171000000000001e-05 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463175+0000", 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8671000000000005e-05 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1878 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 593920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.485150917, 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086539509000000001, 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1878 2024-09-14T07:11:52.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.376824+0000", 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377344+0000", 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00051984700000000004 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377354+0000", 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0135999999999999e-05 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377375+0000", 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0353e-05 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377389+0000", 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3834e-05 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463203+0000", 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.085814968000000005 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463276+0000", 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2492999999999999e-05 2024-09-14T07:11:52.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463364+0000", 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7878000000000001e-05 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1879 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 616448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.484559241, 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.086132710000000001, 2024-09-14T07:11:52.912 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1879 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.377416+0000", 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380512+0000", 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030964030000000002 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380522+0000", 2024-09-14T07:11:52.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8460000000000003e-06 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380544+0000", 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1488000000000002e-05 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380558+0000", 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4549000000000001e-05 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463392+0000", 2024-09-14T07:11:52.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082833614 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463464+0000", 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1907000000000001e-05 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.463549+0000", 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4902999999999997e-05 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.916 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1888 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 819200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.480984498, 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084257514000000006, 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1888 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380991+0000", 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380992+0000", 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5199999999999995e-07 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.380996+0000", 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.408e-06 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381007+0000", 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1358999999999999e-05 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381016+0000", 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6519999999999995e-06 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465089+0000", 2024-09-14T07:11:52.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084072764999999994 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465159+0000", 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0661999999999995e-05 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465248+0000", 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8815999999999998e-05 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1889 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 841728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.48094514499999, 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084499198999999997, 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1889 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381030+0000", 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381031+0000", 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.114e-06 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381035+0000", 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9740000000000004e-06 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381047+0000", 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1844e-05 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381056+0000", 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8580000000000006e-06 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465303+0000", 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084246881999999995 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465419+0000", 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115782 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465529+0000", 2024-09-14T07:11:52.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000110745 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1890 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 864256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.48090736500001, 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084711072999999998, 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1890 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381068+0000", 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381069+0000", 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.54e-07 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381072+0000", 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5530000000000001e-06 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381084+0000", 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1208e-05 2024-09-14T07:11:52.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381092+0000", 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8899999999999996e-06 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465566+0000", 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084473962 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465663+0000", 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7116999999999999e-05 2024-09-14T07:11:52.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465779+0000", 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011548900000000001 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5104.0:1891 3.10 3:08ced707:::rbd_data.116150b87d44.0000000000000006:14 [read 886784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 166.480870662, 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084910016000000005, 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5104", 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1891 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.381105+0000", 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385445+0000", 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043399520000000002 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385500+0000", 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5102000000000002e-05 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385520+0000", 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0142000000000001e-05 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.385537+0000", 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7295999999999998e-05 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465811+0000", 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.080274339 2024-09-14T07:11:52.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.465890+0000", 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8239999999999996e-05 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:09:06.466015+0000", 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000124945 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:934 3.1d 3:b99a04cd:::rbd_data.1161a6fd7139.0000000000000001:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 104.891498728, 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.089202374000000001, 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 934 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970477+0000", 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970478+0000", 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.869e-06 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970488+0000", 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1700000000000003e-06 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970525+0000", 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7166999999999998e-05 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970539+0000", 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4250000000000001e-05 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.970723+0000", 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000184243 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.971042+0000", 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031918000000000002 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059679+0000", 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.088636494999999996 2024-09-14T07:11:52.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:958 3.1d 3:bd0b33be:::rbd_data.1161a6fd7139.000000000000000e:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 104.886826296, 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677687000000001, 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:52.938 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 958 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975149+0000", 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975151+0000", 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6169999999999999e-06 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975158+0000", 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9659999999999999e-06 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975178+0000", 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0092999999999999e-05 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975193+0000", 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4944e-05 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975625+0000", 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043275399999999998 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975933+0000", 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030768899999999999 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059827+0000", 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.083893624 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:961 3.1d 3:b84b7bb7:::rbd_data.1161a6fd7139.0000000000000010:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 104.88674380400001, 2024-09-14T07:11:52.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084712299000000005, 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 961 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975231+0000", 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975233+0000", 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3039999999999999e-06 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975239+0000", 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2530000000000001e-06 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975265+0000", 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5930000000000001e-05 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975279+0000", 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4355999999999999e-05 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977075+0000", 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017954550000000001 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.977383+0000", 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030783999999999998 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059944+0000", 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.082561160999999994 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:973 3.1d 3:bf667fbf:::rbd_data.1161a6fd7139.0000000000000017:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:52.946 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 104.886582683, 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084677382999999995, 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 973 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975393+0000", 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975394+0000", 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4640000000000001e-06 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975401+0000", 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975424+0000", 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3507e-05 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975439+0000", 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4756000000000001e-05 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978442+0000", 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030033820000000002 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978767+0000", 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032446299999999999 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.060070+0000", 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081303084999999997 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.5755.0:980 3.1c 3:39c65fc5:::rbd_data.1161a6fd7139.000000000000001c:head [stat,delete] snapc 23=[23] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-09-14T07:11:52.950 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 104.886405775, 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.084274022000000004, 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "continuous": false, 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.5755", 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.145:0/2932306107", 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 980 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-14T07:11:52.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975570+0000", 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975571+0000", 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4169999999999999e-06 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975578+0000", 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7499999999999997e-06 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975605+0000", 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7002e-05 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.975620+0000", 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5044e-05 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-14T07:11:52.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978471+0000", 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002851714 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:07.978752+0000", 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028063300000000002 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-14T07:10:08.059844+0000", 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.081091462000000003 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-14T07:11:52.954 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-14T07:11:52.955 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:11:52.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:52.956 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:52.953+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:53.057 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:53.053+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:53.088 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:53.088 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:53.089 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:53.089 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:53.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:53.191 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:53.189+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:53.258 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:53.253+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:53.272 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:53.272 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:53.272 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:53.272 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:53.272 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:53.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:53.359 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:53.353+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(osd.5.0:5567 3.17 e210/209 3:ebfbd17b:::rbd_data.11615ad0c92d.0000000000000079:15 v 210'234, mlcod=203'231)", 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.821837+0000", 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.636557164999999, 2024-09-14T07:11:53.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0073217239999999999, 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821837+0000", 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821837+0000", 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821838+0000", 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2699999999999998e-07 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821854+0000", 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.645e-05 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821865+0000", 2024-09-14T07:11:53.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0400999999999999e-05 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821872+0000", 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7060000000000004e-06 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821921+0000", 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8279999999999999e-05 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.821940+0000", 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8938999999999999e-05 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.829114+0000", 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0071739780000000001 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.829159+0000", 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5142999999999998e-05 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.465 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop_reply(osd.0.0:1 3.1f e210/209 ondisk, result = 0)", 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.823819+0000", 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.634574881000001, 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053448030000000004, 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823819+0000", 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823819+0000", 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823820+0000", 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-07 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823825+0000", 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823861+0000", 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6134000000000003e-05 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823869+0000", 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9810000000000003e-06 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.829139+0000", 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0052702139999999996 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.829149+0000", 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8199999999999992e-06 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.829164+0000", 2024-09-14T07:11:53.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5287000000000001e-05 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(osd.4.0:3588 3.1a e210/209 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:b v 210'378, mlcod=201'375)", 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.862497+0000", 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.595897364000001, 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063413200000000001, 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.862497+0000", 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.862497+0000", 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.862497+0000", 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.862503+0000", 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6570000000000002e-06 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.862514+0000", 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1354e-05 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.862524+0000", 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8619999999999998e-06 2024-09-14T07:11:53.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.862995+0000", 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047094299999999998 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.863011+0000", 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5784e-05 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.868809+0000", 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0057982370000000004 2024-09-14T07:11:53.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.868838+0000", 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8864e-05 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(osd.1.0:2752 3.b e210/209 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:b v 210'997, mlcod=208'994)", 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.892442+0000", 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.565951718999999, 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004315864, 2024-09-14T07:11:53.474 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892442+0000", 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892442+0000", 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892443+0000", 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5600000000000005e-07 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892449+0000", 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3230000000000003e-06 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892460+0000", 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0351e-05 2024-09-14T07:11:53.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892468+0000", 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.259e-06 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892516+0000", 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7667000000000003e-05 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.892533+0000", 2024-09-14T07:11:53.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7331999999999999e-05 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.896714+0000", 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041813509999999998 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.896758+0000", 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3924999999999997e-05 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(osd.3.0:3712 3.15 e210/209 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:b v 210'493, mlcod=199'490)", 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.937633+0000", 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.520761383, 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046562720000000004, 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937633+0000", 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937633+0000", 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937633+0000", 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2e-07 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937637+0000", 2024-09-14T07:11:53.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1559999999999997e-06 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937645+0000", 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6739999999999997e-06 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937650+0000", 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8770000000000001e-06 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937694+0000", 2024-09-14T07:11:53.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3611e-05 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.937714+0000", 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0024999999999999e-05 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.942222+0000", 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045084349999999999 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.942289+0000", 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6973999999999999e-05 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:28387 3.6 e211/210 3:6355d838:::rbd_data.11615839fdc9.0000000000000007:head v 211'682, mlcod=207'680)", 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:43.917944+0000", 2024-09-14T07:11:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5404499250000008, 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003054331, 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.917944+0000", 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.917944+0000", 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.917946+0000", 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.389e-06 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.917954+0000", 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2689999999999999e-06 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.917974+0000", 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0298000000000001e-05 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.917985+0000", 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1025e-05 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.918152+0000", 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016695999999999999 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.918214+0000", 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1748999999999994e-05 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.920946+0000", 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027319380000000002 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.920999+0000", 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2703e-05 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:28390 3.b e211/209 3:d77b81c3:::rbd_data.11615839fdc9.000000000000000a:head v 211'1004, mlcod=210'1001)", 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:43.922144+0000", 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5362504930000007, 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082755550000000004, 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922144+0000", 2024-09-14T07:11:53.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922144+0000", 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922146+0000", 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.933e-06 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922159+0000", 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.382e-05 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922195+0000", 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5916e-05 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922215+0000", 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9378999999999999e-05 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922342+0000", 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012701300000000001 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.922364+0000", 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2583e-05 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.930326+0000", 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079620300000000001 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.930419+0000", 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2880999999999994e-05 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:28392 3.15 e211/209 3:ac3e59ab:::rbd_data.11615839fdc9.000000000000000c:head v 211'504, mlcod=210'501)", 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:43.923499+0000", 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.534895508, 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040231329999999999, 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923499+0000", 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923499+0000", 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923499+0000", 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8599999999999997e-07 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923509+0000", 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5140000000000007e-06 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923537+0000", 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8289e-05 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923557+0000", 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9933e-05 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923638+0000", 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0717999999999995e-05 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923663+0000", 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5151e-05 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.927394+0000", 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037308189999999998 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.927522+0000", 2024-09-14T07:11:53.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000127923 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:28391 3.3 e211/210 3:c712f81b:::rbd_data.11615839fdc9.000000000000000b:head v 211'469, mlcod=211'467)", 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:43.923616+0000", 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5347777540000003, 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007151704, 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.497 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923616+0000", 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923616+0000", 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923618+0000", 2024-09-14T07:11:53.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4440000000000001e-06 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923630+0000", 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.219e-05 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923667+0000", 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7339000000000003e-05 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923685+0000", 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7382999999999999e-05 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923865+0000", 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000180206 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.923929+0000", 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3577999999999993e-05 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.930734+0000", 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0068052620000000003 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.930768+0000", 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4301999999999998e-05 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28393 3.1f 3:fa24114e:::rbd_data.11615839fdc9.000000000000000d:head [delete] snapc b=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:43.924018+0000", 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5343765719999993, 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065043649999999998, 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28393 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924018+0000", 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924018+0000", 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924020+0000", 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5629999999999999e-06 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924030+0000", 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0213e-05 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924057+0000", 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6682999999999998e-05 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924087+0000", 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9519e-05 2024-09-14T07:11:53.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924199+0000", 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000112952 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924298+0000", 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8203999999999997e-05 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924436+0000", 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013816300000000001 2024-09-14T07:11:53.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.928123+0000", 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003686819 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.930423+0000", 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023002119999999998 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.930474+0000", 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0720000000000002e-05 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.930522+0000", 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8316999999999998e-05 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4449.0:28394 3.1a e211/209 3:5b03a6e0:::rbd_data.11615839fdc9.000000000000000e:head v 211'385, mlcod=210'382)", 2024-09-14T07:11:53.507 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:43.924252+0000", 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5341420669999994, 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044968400000000002, 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924252+0000", 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924252+0000", 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924253+0000", 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3239999999999999e-06 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924266+0000", 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2772000000000001e-05 2024-09-14T07:11:53.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924289+0000", 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2952999999999999e-05 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924309+0000", 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9550000000000001e-05 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924489+0000", 2024-09-14T07:11:53.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018002 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.924544+0000", 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5182000000000001e-05 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.928623+0000", 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040795120000000004 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.928749+0000", 2024-09-14T07:11:53.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012552700000000001 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(osd.1.0:2774 3.b e212/209 3:d0e6946d:::rbd_data.116167e24a2d.0000000000000009:a v 212'1006, mlcod=210'1003)", 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:44.810266+0000", 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6481283080000004, 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002747635, 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810266+0000", 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810266+0000", 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810267+0000", 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.268e-06 2024-09-14T07:11:53.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810278+0000", 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1066e-05 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810303+0000", 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5239e-05 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810318+0000", 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4928e-05 2024-09-14T07:11:53.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810409+0000", 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0660000000000003e-05 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.810435+0000", 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5527000000000001e-05 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.812931+0000", 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002496655 2024-09-14T07:11:53.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:44.813013+0000", 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2292000000000004e-05 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28745 3.10 3:0ac53f0d:::rbd_data.11616dafc67f.000000000000000c:head [delete] snapc 9=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:47.784756+0000", 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.6736382750000001, 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.032418348, 2024-09-14T07:11:53.516 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28745 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784756+0000", 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.518 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:53.457+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784756+0000", 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784759+0000", 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.473e-06 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784776+0000", 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7059999999999999e-05 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784824+0000", 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7296000000000003e-05 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784850+0000", 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5945999999999999e-05 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.785043+0000", 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019290999999999999 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op must wait for map", 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.785064+0000", 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1826000000000001e-05 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.791934+0000", 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006869361 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.791959+0000", 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4853000000000001e-05 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.814767+0000", 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0228087990000001 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.815050+0000", 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028266700000000001 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.815246+0000", 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000196314 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.816480+0000", 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012334939999999999 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.816953+0000", 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047326900000000001 2024-09-14T07:11:53.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.817037+0000", 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3490000000000002e-05 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.817174+0000", 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013758999999999999 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(osd.3.0:3819 3.15 e220/209 3:aa0a31e1:::rbd_data.116148d60084.000000000000000d:6 v 220'512, mlcod=217'509)", 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:52.791982+0000", 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.666411788, 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0066411660000000004, 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.791982+0000", 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.791982+0000", 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.791983+0000", 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8699999999999995e-07 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.791993+0000", 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0293e-05 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.792019+0000", 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5333e-05 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.792044+0000", 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5839000000000001e-05 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.792217+0000", 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000172251 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.792285+0000", 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8229e-05 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.798567+0000", 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062820649999999999 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.798624+0000", 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6569e-05 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:29156 3.6 3:668efea6:::rbd_object_map.116148d60084:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e220)", 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:52.811543+0000", 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.64685147799999998, 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036440859999999999, 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 29156 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.811543+0000", 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.811543+0000", 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.811545+0000", 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-09-14T07:11:53.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.811556+0000", 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0986e-05 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.811592+0000", 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5599999999999998e-05 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.811609+0000", 2024-09-14T07:11:53.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6705999999999999e-05 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.811768+0000", 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015918400000000001 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.812028+0000", 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026043600000000001 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:53.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.812422+0000", 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00039360600000000001 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.813828+0000", 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001406324 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.814964+0000", 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001136214 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.815068+0000", 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103862 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.815187+0000", 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118468 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:29177 3.6 3:668efea6:::rbd_object_map.116148d60084:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e220)", 2024-09-14T07:11:53.534 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:52.874109+0000", 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.58428518399999996, 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029719070000000002, 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 29177 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874109+0000", 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874109+0000", 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874112+0000", 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.667e-06 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874124+0000", 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2386e-05 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874155+0000", 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1415999999999999e-05 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874172+0000", 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6503000000000001e-05 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874294+0000", 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000122528 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874450+0000", 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015554699999999999 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.874754+0000", 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030354200000000002 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.876066+0000", 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013124549999999999 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.876911+0000", 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084482600000000002 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.877003+0000", 2024-09-14T07:11:53.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2439e-05 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.877081+0000", 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7597999999999998e-05 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:29185 3.6 3:668efea6:::rbd_object_map.116148d60084:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e220)", 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:52.882147+0000", 2024-09-14T07:11:53.540 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.57624733800000005, 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029053680000000002, 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 29185 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882147+0000", 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882147+0000", 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882150+0000", 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.05e-06 2024-09-14T07:11:53.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882160+0000", 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0406e-05 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882186+0000", 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6129999999999999e-05 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882200+0000", 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3626e-05 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882330+0000", 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012982799999999999 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882481+0000", 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000150724 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.882764+0000", 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028353900000000002 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.884301+0000", 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015365940000000001 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.884891+0000", 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059009999999999998 2024-09-14T07:11:53.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.884982+0000", 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1112000000000005e-05 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.885052+0000", 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0259000000000006e-05 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:29205 3.6 3:668efea6:::rbd_object_map.116148d60084:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e220)", 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:52.918985+0000", 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.53940932799999997, 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027959270000000001, 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 29205 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.918985+0000", 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.918985+0000", 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.918987+0000", 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0140000000000001e-06 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.918998+0000", 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0926e-05 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.919027+0000", 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-05 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.919043+0000", 2024-09-14T07:11:53.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5183e-05 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.919163+0000", 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120308 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.919317+0000", 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015384 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.919599+0000", 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000282315 2024-09-14T07:11:53.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.920708+0000", 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001109236 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.921599+0000", 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00089043100000000001 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.921695+0000", 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6842000000000006e-05 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.921781+0000", 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5241999999999995e-05 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:29209 3.6 3:668efea6:::rbd_object_map.116148d60084:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e220)", 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:52.925180+0000", 2024-09-14T07:11:53.552 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.53321391500000004, 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027698269999999999, 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 29209 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925180+0000", 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925180+0000", 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925182+0000", 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2299999999999998e-06 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925195+0000", 2024-09-14T07:11:53.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2903e-05 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925227+0000", 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1705999999999998e-05 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925242+0000", 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5277e-05 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925384+0000", 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014186400000000001 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925535+0000", 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000151218 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.925815+0000", 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00027996100000000001 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.927108+0000", 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012926019999999999 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.927785+0000", 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00067716499999999997 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.927877+0000", 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2299000000000005e-05 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:52.927950+0000", 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2601999999999994e-05 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:29316 3.1b 3:d8c829cf:::rbd_object_map.116121ff13e0.0000000000000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e220)", 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:53.105149+0000", 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.35324563399999998, 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030644420000000001, 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:53.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 29316 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105149+0000", 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105149+0000", 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105150+0000", 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1969999999999999e-06 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105156+0000", 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1979999999999998e-06 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105173+0000", 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7139999999999999e-05 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105185+0000", 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2268e-05 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105307+0000", 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012160299999999999 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105461+0000", 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015393500000000001 2024-09-14T07:11:53.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.105660+0000", 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000199493 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.106928+0000", 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00126812 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.108017+0000", 2024-09-14T07:11:53.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010880639999999999 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.108112+0000", 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5902000000000004e-05 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:53.108213+0000", 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100522 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:53.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:53.564 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:53.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:11:53.565 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:53.565+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:53.667 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:53.665+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:53.694 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-09-14T07:11:53.694 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:53.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:11:53.768 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:53.765+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:53.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:11:53.869 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:53.865+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:53.889 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-09-14T07:11:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-09-14T07:11:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-09-14T07:11:53.889 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:53.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:11:53.996 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:53.993+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:54.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:11:54.071 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:54.065+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:54.081 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4703.0:374 3.8 e46/25 3:1243660c:::rbd_data.1161ff1af5c4.0000000000000000:head v 46'67, mlcod=43'64)", 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 212.40037764300001, 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.134766727, 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679785+0000", 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.679788+0000", 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693347+0000", 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013558966 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693388+0000", 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0744000000000002e-05 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693406+0000", 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8306000000000001e-05 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693598+0000", 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000192237 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.693641+0000", 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3294999999999998e-05 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814485+0000", 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120843772 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:08:21.814552+0000", 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6459999999999997e-05 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9251 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00924072800001, 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120425961, 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9251 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070922+0000", 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070923+0000", 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8199999999999999e-07 2024-09-14T07:11:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070936+0000", 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3767e-05 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070949+0000", 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2106999999999999e-05 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070957+0000", 2024-09-14T07:11:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8219999999999996e-06 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191333+0000", 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120375484 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191348+0000", 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4999e-05 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9252 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00917892000001, 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120421522, 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9252 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070984+0000", 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070985+0000", 2024-09-14T07:11:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4099999999999997e-07 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070987+0000", 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.070997+0000", 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7999999999999993e-06 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071002+0000", 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191394+0000", 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120391569 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191405+0000", 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1214e-05 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9254 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.009144604, 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446652, 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9254 2024-09-14T07:11:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071018+0000", 2024-09-14T07:11:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071019+0000", 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071020+0000", 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071027+0000", 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6649999999999997e-06 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071031+0000", 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1670000000000001e-06 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191451+0000", 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120419816 2024-09-14T07:11:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191465+0000", 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3709e-05 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9255 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00912593300001, 2024-09-14T07:11:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120473359, 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9255 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071037+0000", 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071046+0000", 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7360000000000007e-06 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071051+0000", 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5310000000000001e-06 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071055+0000", 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0860000000000004e-06 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191502+0000", 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120446398 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191510+0000", 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1920000000000005e-06 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9257 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00910189699999, 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120511618, 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9257 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071061+0000", 2024-09-14T07:11:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071063+0000", 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5400000000000001e-06 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071067+0000", 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7260000000000002e-06 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071075+0000", 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3490000000000003e-06 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191563+0000", 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120488761 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191572+0000", 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8680000000000005e-06 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9258 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00908186300001, 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120536985, 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9258 2024-09-14T07:11:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071081+0000", 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071083+0000", 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.494e-06 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071087+0000", 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:11:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071091+0000", 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6559999999999998e-06 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191609+0000", 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12051769499999999 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191618+0000", 2024-09-14T07:11:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0960000000000007e-06 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9260 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.009067028, 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120556968, 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9260 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071096+0000", 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071097+0000", 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071102+0000", 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.493e-06 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071106+0000", 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.664e-06 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191646+0000", 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120540494 2024-09-14T07:11:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191653+0000", 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9262 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00905240200001, 2024-09-14T07:11:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120577881, 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9262 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071110+0000", 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071111+0000", 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071112+0000", 2024-09-14T07:11:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071117+0000", 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6059999999999996e-06 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071120+0000", 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6789999999999998e-06 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191680+0000", 2024-09-14T07:11:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120559866 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191688+0000", 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9263 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.009037901, 2024-09-14T07:11:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120593641, 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9263 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071125+0000", 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071127+0000", 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.525e-06 2024-09-14T07:11:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071132+0000", 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5049999999999998e-06 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071138+0000", 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191712+0000", 2024-09-14T07:11:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.1205745 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191718+0000", 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9264 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00902014600001, 2024-09-14T07:11:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120609748, 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9264 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071142+0000", 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071143+0000", 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071144+0000", 2024-09-14T07:11:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4580000000000001e-06 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071149+0000", 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071153+0000", 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.681e-06 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191745+0000", 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120591593 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191752+0000", 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7179999999999996e-06 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9265 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00900525200001, 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120613711, 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9265 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071157+0000", 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071158+0000", 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071159+0000", 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.322e-06 2024-09-14T07:11:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071164+0000", 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071167+0000", 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191766+0000", 2024-09-14T07:11:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120598371 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191771+0000", 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9267 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.008991141, 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062002600000001, 2024-09-14T07:11:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9267 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071171+0000", 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071172+0000", 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071179+0000", 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2450000000000001e-06 2024-09-14T07:11:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071184+0000", 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1590000000000001e-06 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071188+0000", 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9659999999999998e-06 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191784+0000", 2024-09-14T07:11:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12059546 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191792+0000", 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8560000000000007e-06 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9268 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.00896921200001, 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120628508, 2024-09-14T07:11:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9268 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071193+0000", 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071194+0000", 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071200+0000", 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7969999999999997e-06 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071205+0000", 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.896e-06 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071209+0000", 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9430000000000002e-06 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191816+0000", 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120606282 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191822+0000", 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2550000000000003e-06 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5104.0:9269 3.6 3.a6fe3d26 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 167.008947901, 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.120640757, 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5104", 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/1046830123", 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9269 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071215+0000", 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071217+0000", 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:11:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071221+0000", 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.071225+0000", 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7419999999999999e-06 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191848+0000", 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.12062262999999999 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:07.191855+0000", 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8099999999999998e-06 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.5125.0:243 3.6 3:67acb7bd:::rbd_data.11615ad0c92d.0000000000000000:15 [read 6144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:11:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 162.721597, 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166868113, 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.5125", 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/3733343923", 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 243 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358566+0000", 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358567+0000", 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2500000000000004e-07 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358571+0000", 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6330000000000004e-06 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358585+0000", 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3402e-05 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358592+0000", 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:11:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358662+0000", 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0182e-05 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:11.358672+0000", 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0686000000000001e-05 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525031+0000", 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.166358247 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525268+0000", 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000237865 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:09:12.525434+0000", 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016526299999999999 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:22366 3.8 3:12648009:::rbd_object_map.116130782be1:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.219928611, 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93379210400000001, 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 22366 2024-09-14T07:11:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860234+0000", 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860235+0000", 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.02e-07 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860239+0000", 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4859999999999999e-06 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860250+0000", 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1069e-05 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860256+0000", 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1710000000000001e-06 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860313+0000", 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7720999999999998e-05 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:57.860322+0000", 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0110000000000006e-06 2024-09-14T07:11:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791503+0000", 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.93118071000000002 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791695+0000", 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019195799999999999 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.791891+0000", 2024-09-14T07:11:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019566200000000001 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793017+0000", 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126525 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793819+0000", 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080132300000000005 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.793906+0000", 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7374999999999996e-05 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:10:58.794026+0000", 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000120191 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28319 3.6 3:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e210)", 2024-09-14T07:11:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.256266579, 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.976578579, 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28319 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823896+0000", 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823897+0000", 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4600000000000004e-07 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823900+0000", 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4910000000000001e-06 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823908+0000", 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5229999999999998e-06 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823912+0000", 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.143e-06 2024-09-14T07:11:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823975+0000", 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2892999999999996e-05 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:42.823987+0000", 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2215000000000001e-05 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.800096+0000", 2024-09-14T07:11:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.97610919500000004 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:43.800475+0000", 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00037847299999999999 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28737 3.16 3:6b13d953:::rbd_data.11616dafc67f.0000000000000000:head [delete] snapc 9=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:47.784201+0000", 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.2959611930000001, 2024-09-14T07:11:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.036575588, 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28737 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784201+0000", 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784201+0000", 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784205+0000", 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6550000000000002e-06 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784220+0000", 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4717e-05 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784257+0000", 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6748000000000001e-05 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784273+0000", 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6844e-05 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784430+0000", 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015652999999999999 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op must wait for map", 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784441+0000", 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0701e-05 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.790423+0000", 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0059825349999999998 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.790431+0000", 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2779999999999993e-06 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.818805+0000", 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.028374034 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.819019+0000", 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00021321799999999999 2024-09-14T07:11:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3,5", 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.819132+0000", 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011290000000000001 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.819891+0000", 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000758985 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.820543+0000", 2024-09-14T07:11:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00065237300000000004 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.820642+0000", 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8669e-05 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.820681+0000", 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9156999999999998e-05 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.820777+0000", 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6243999999999995e-05 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4449.0:28746 3.e 3:75d6b02c:::rbd_data.11616dafc67f.000000000000000d:head [delete] snapc 9=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-09-14T07:11:47.784321+0000", 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.2958416929999999, 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0385441550000001, 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-09-14T07:11:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4449", 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.145:0/2959918752", 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28746 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784321+0000", 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-09-14T07:11:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784321+0000", 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784322+0000", 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4300000000000002e-07 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784326+0000", 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4870000000000004e-06 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-09-14T07:11:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784338+0000", 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2086000000000001e-05 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784346+0000", 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1040000000000007e-06 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784543+0000", 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019653699999999999 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op must wait for map", 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.784562+0000", 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9437e-05 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.790233+0000", 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056702660000000002 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:47.790244+0000", 2024-09-14T07:11:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0987e-05 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.819825+0000", 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0295814480000001 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.820308+0000", 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00048255800000000001 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4,5", 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.820580+0000", 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000272551 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.821597+0000", 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010166089999999999 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.822032+0000", 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043466000000000002 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.822175+0000", 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014372500000000001 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.822424+0000", 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024873899999999999 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-09-14T07:11:48.822865+0000", 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000441118 2024-09-14T07:11:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-14T07:11:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-14T07:11:54.177 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:11:54.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:11:54.179 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-09-14T07:11:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:11:54.179 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-09-14T07:11:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:11:54.180 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:54.173+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:54.194 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-09-14T07:11:54.194 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:11:54.215 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-09-14T07:11:54.215 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:11:54.236 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-09-14T07:11:54.237 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:11:54.257 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-09-14T07:11:54.258 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:11:54.279 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:54.277+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:54.324 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:54.324 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:54.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:11:54.380 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:54.377+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:54.463 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:54.463 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:54.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:11:54.480 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:54.477+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:54.581 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:54.577+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:54.604 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:54.604 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:54.620 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:11:54.682 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-14T07:11:54.677+0000 7f97ca32a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25266) UID: 0 2024-09-14T07:11:54.752 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:54.752 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:54.766 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:11:54.808 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:54.805+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:11:54.883 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:54.881+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:54.935 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:54.935 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:54.950 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:11:54.996 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:54.993+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:55.082 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:55.083 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:55.085 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-09-14T07:11:55.081+0000 7fa8fe450640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25150) UID: 0 2024-09-14T07:11:55.099 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:11:55.186 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:55.181+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:55.265 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:55.265 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:55.280 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:11:55.332 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:55.329+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:55.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:11:55.401 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:55.401 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:55.415 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:11:55.431 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:55.429+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:55.487 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-09-14T07:11:55.485+0000 7f68a1b6b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25148) UID: 0 2024-09-14T07:11:55.585 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:55.585 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:55.588 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:55.585+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:55.599 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:11:55.692 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:55.689+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:55.703 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:11:55.722 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-14T07:11:55.271915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":127440249,"num_objects":339,"num_object_clones":9,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":254,"num_read":340672,"num_read_kb":1115663,"num_write":21951,"num_write_kb":331483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19302,"ondisk_log_size":19302,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":350352,"kb_used_data":18104,"kb_used_omap":35,"kb_used_meta":331996,"kb_avail":565880688,"statfs":{"total":579820584960,"available":579461824512,"internally_reserved":0,"allocated":18538496,"data_stored":25278223,"data_compressed":1335860,"data_compressed_allocated":2207744,"data_compressed_original":18468864,"omap_allocated":36329,"internal_metadata":339964439},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1277623,"num_objects":-6,"num_object_clones":-1,"num_object_copies":-12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6,"num_whiteouts":-1,"num_read":34,"num_read_kb":31,"num_write":13,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001177"},"pg_stats":[{"pgid":"3.1f","version":"217'381","reported_seq":6950,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820211+0000","last_change":"2024-09-14T07:11:49.823797+0000","last_active":"2024-09-14T07:11:50.820211+0000","last_peered":"2024-09-14T07:11:50.820211+0000","last_clean":"2024-09-14T07:11:50.820211+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:11:50.820211+0000","last_undegraded":"2024-09-14T07:11:50.820211+0000","last_fullsized":"2024-09-14T07:11:50.820211+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:04:11.679482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081229,"stat_sum":{"num_bytes":16,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6401,"num_read_kb":50527,"num_write":555,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1e","version":"217'720","reported_seq":12973,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001526+0000","last_change":"2024-09-14T07:11:49.832703+0000","last_active":"2024-09-14T07:11:50.001526+0000","last_peered":"2024-09-14T07:11:50.001526+0000","last_clean":"2024-09-14T07:11:50.001526+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:11:50.001526+0000","last_undegraded":"2024-09-14T07:11:50.001526+0000","last_fullsized":"2024-09-14T07:11:50.001526+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:57:12.557671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077816990000000004,"stat_sum":{"num_bytes":5972628,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":14175,"num_read_kb":59693,"num_write":943,"num_write_kb":21289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1d","version":"217'1602","reported_seq":16581,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005381+0000","last_change":"2024-09-14T07:11:49.834031+0000","last_active":"2024-09-14T07:11:50.005381+0000","last_peered":"2024-09-14T07:11:50.005381+0000","last_clean":"2024-09-14T07:11:50.005381+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:11:50.005381+0000","last_undegraded":"2024-09-14T07:11:50.005381+0000","last_fullsized":"2024-09-14T07:11:50.005381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1602,"log_dups_size":0,"ondisk_log_size":1602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:22:12.091166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072780320000000003,"stat_sum":{"num_bytes":4808704,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":20893,"num_read_kb":36306,"num_write":1696,"num_write_kb":5676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1c","version":"217'501","reported_seq":20791,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.895045+0000","last_change":"2024-09-14T07:11:49.826028+0000","last_active":"2024-09-14T07:11:49.895045+0000","last_peered":"2024-09-14T07:11:49.895045+0000","last_clean":"2024-09-14T07:11:49.895045+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:11:49.895045+0000","last_undegraded":"2024-09-14T07:11:49.895045+0000","last_fullsized":"2024-09-14T07:11:49.895045+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:25.506830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001223399,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21030,"num_read_kb":46630,"num_write":501,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1b","version":"214'572","reported_seq":8666,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819776+0000","last_change":"2024-09-14T07:11:49.823068+0000","last_active":"2024-09-14T07:11:50.819776+0000","last_peered":"2024-09-14T07:11:50.819776+0000","last_clean":"2024-09-14T07:11:50.819776+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:11:50.819776+0000","last_undegraded":"2024-09-14T07:11:50.819776+0000","last_fullsized":"2024-09-14T07:11:50.819776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:06:03.189849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015995510000000001,"stat_sum":{"num_bytes":19174036,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":8775,"num_read_kb":22847,"num_write":713,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1a","version":"217'391","reported_seq":30324,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-14T07:11:52.008839+0000","last_change":"2024-09-14T07:11:51.838708+0000","last_active":"2024-09-14T07:11:52.008839+0000","last_peered":"2024-09-14T07:11:52.008839+0000","last_clean":"2024-09-14T07:11:52.008839+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:52.008839+0000","last_undegraded":"2024-09-14T07:11:52.008839+0000","last_fullsized":"2024-09-14T07:11:52.008839+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:48.578400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022117619999999999,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":29657,"num_read_kb":50539,"num_write":415,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1d"}]},{"pgid":"3.19","version":"217'1338","reported_seq":10587,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822350+0000","last_change":"2024-09-14T07:11:49.822350+0000","last_active":"2024-09-14T07:11:49.822350+0000","last_peered":"2024-09-14T07:11:49.822350+0000","last_clean":"2024-09-14T07:11:49.822350+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:11:49.822350+0000","last_undegraded":"2024-09-14T07:11:49.822350+0000","last_fullsized":"2024-09-14T07:11:49.822350+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1338,"log_dups_size":0,"ondisk_log_size":1338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:39.514580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042770159999999998,"stat_sum":{"num_bytes":2836518,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":14561,"num_read_kb":35933,"num_write":2117,"num_write_kb":9607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.18","version":"218'427","reported_seq":5411,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.949888+0000","last_change":"2024-09-14T07:11:49.822391+0000","last_active":"2024-09-14T07:11:50.949888+0000","last_peered":"2024-09-14T07:11:50.949888+0000","last_clean":"2024-09-14T07:11:50.949888+0000","last_became_active":"2024-09-14T07:11:42.792094+0000","last_became_peered":"2024-09-14T07:11:42.792094+0000","last_unstale":"2024-09-14T07:11:50.949888+0000","last_undegraded":"2024-09-14T07:11:50.949888+0000","last_fullsized":"2024-09-14T07:11:50.949888+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:42:01.973449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096171600000000005,"stat_sum":{"num_bytes":270384,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":4861,"num_read_kb":15257,"num_write":485,"num_write_kb":15533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.17","version":"217'242","reported_seq":3834,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.005660+0000","last_change":"2024-09-14T07:11:49.834634+0000","last_active":"2024-09-14T07:11:50.005660+0000","last_peered":"2024-09-14T07:11:50.005660+0000","last_clean":"2024-09-14T07:11:50.005660+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:11:50.005660+0000","last_undegraded":"2024-09-14T07:11:50.005660+0000","last_fullsized":"2024-09-14T07:11:50.005660+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:55.293205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.16","version":"216'640","reported_seq":9998,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.573145+0000","last_change":"2024-09-14T07:11:50.307883+0000","last_active":"2024-09-14T07:11:50.307883+0000","last_peered":"2024-09-14T07:11:49.573145+0000","last_clean":"2024-09-14T07:11:43.788063+0000","last_became_active":"2024-09-14T07:11:48.815066+0000","last_became_peered":"2024-09-14T07:11:48.815066+0000","last_unstale":"2024-09-14T07:11:49.573145+0000","last_undegraded":"2024-09-14T07:11:49.573145+0000","last_fullsized":"2024-09-14T07:11:49.573145+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084910960000000001,"stat_sum":{"num_bytes":1368064,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":738,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.15","version":"217'509","reported_seq":6975,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.833831+0000","last_change":"2024-09-14T07:11:49.833831+0000","last_active":"2024-09-14T07:11:49.833831+0000","last_peered":"2024-09-14T07:11:49.833831+0000","last_clean":"2024-09-14T07:11:49.833831+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:11:49.833831+0000","last_undegraded":"2024-09-14T07:11:49.833831+0000","last_fullsized":"2024-09-14T07:11:49.833831+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:17.752405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075586000000000004,"stat_sum":{"num_bytes":4245041,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":7401,"num_read_kb":47217,"num_write":522,"num_write_kb":16662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.14","version":"217'750","reported_seq":11076,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.933900+0000","last_change":"2024-09-14T07:11:49.830868+0000","last_active":"2024-09-14T07:11:49.933900+0000","last_peered":"2024-09-14T07:11:49.933900+0000","last_clean":"2024-09-14T07:11:49.933900+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:11:49.933900+0000","last_undegraded":"2024-09-14T07:11:49.933900+0000","last_fullsized":"2024-09-14T07:11:49.933900+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:21.610420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066874480000000004,"stat_sum":{"num_bytes":12418560,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":12254,"num_read_kb":39191,"num_write":924,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.7","version":"0'0","reported_seq":429,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826029+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:11:49.826029+0000","last_peered":"2024-09-14T07:11:49.826029+0000","last_clean":"2024-09-14T07:11:49.826029+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:11:49.826029+0000","last_undegraded":"2024-09-14T07:11:49.826029+0000","last_fullsized":"2024-09-14T07:11:49.826029+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"218'692","reported_seq":12786,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820180+0000","last_change":"2024-09-14T07:11:49.823661+0000","last_active":"2024-09-14T07:11:50.820180+0000","last_peered":"2024-09-14T07:11:50.820180+0000","last_clean":"2024-09-14T07:11:50.820180+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:11:50.820180+0000","last_undegraded":"2024-09-14T07:11:50.820180+0000","last_fullsized":"2024-09-14T07:11:50.820180+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:52:36.850781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001773719,"stat_sum":{"num_bytes":129628,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":14012,"num_read_kb":35141,"num_write":738,"num_write_kb":5113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.819091+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:11:49.819091+0000","last_peered":"2024-09-14T07:11:49.819091+0000","last_clean":"2024-09-14T07:11:49.819091+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:11:49.819091+0000","last_undegraded":"2024-09-14T07:11:49.819091+0000","last_fullsized":"2024-09-14T07:11:49.819091+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"217'658","reported_seq":8317,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.001581+0000","last_change":"2024-09-14T07:11:49.820591+0000","last_active":"2024-09-14T07:11:50.001581+0000","last_peered":"2024-09-14T07:11:50.001581+0000","last_clean":"2024-09-14T07:11:50.001581+0000","last_became_active":"2024-09-14T07:10:57.225048+0000","last_became_peered":"2024-09-14T07:10:57.225048+0000","last_unstale":"2024-09-14T07:11:50.001581+0000","last_undegraded":"2024-09-14T07:11:50.001581+0000","last_fullsized":"2024-09-14T07:11:50.001581+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:30:34.825320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025417880000000001,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":701,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.5","version":"0'0","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.827578+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:11:49.827578+0000","last_peered":"2024-09-14T07:11:49.827578+0000","last_clean":"2024-09-14T07:11:49.827578+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:11:49.827578+0000","last_undegraded":"2024-09-14T07:11:49.827578+0000","last_fullsized":"2024-09-14T07:11:49.827578+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"216'686","reported_seq":6671,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.821973+0000","last_change":"2024-09-14T07:11:49.822694+0000","last_active":"2024-09-14T07:11:49.821973+0000","last_peered":"2024-09-14T07:11:49.821973+0000","last_clean":"2024-09-14T07:11:49.821973+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:11:49.821973+0000","last_undegraded":"2024-09-14T07:11:49.821973+0000","last_fullsized":"2024-09-14T07:11:49.821973+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:34:36.123624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042077470000000004,"stat_sum":{"num_bytes":131072,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":862,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.818494+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:11:49.818494+0000","last_peered":"2024-09-14T07:11:49.818494+0000","last_clean":"2024-09-14T07:11:49.818494+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:11:49.818494+0000","last_undegraded":"2024-09-14T07:11:49.818494+0000","last_fullsized":"2024-09-14T07:11:49.818494+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"217'750","reported_seq":4024,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.994308+0000","last_change":"2024-09-14T07:11:49.834054+0000","last_active":"2024-09-14T07:11:49.994308+0000","last_peered":"2024-09-14T07:11:49.994308+0000","last_clean":"2024-09-14T07:11:49.994308+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:11:49.994308+0000","last_undegraded":"2024-09-14T07:11:49.994308+0000","last_fullsized":"2024-09-14T07:11:49.994308+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:44:07.647554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006901735,"stat_sum":{"num_bytes":5210128,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":5230,"num_read_kb":19720,"num_write":1110,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.3","version":"219'498","reported_seq":10071,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-14T07:11:51.992052+0000","last_change":"2024-09-14T07:11:51.839726+0000","last_active":"2024-09-14T07:11:51.992052+0000","last_peered":"2024-09-14T07:11:51.992052+0000","last_clean":"2024-09-14T07:11:51.992052+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:11:51.992052+0000","last_undegraded":"2024-09-14T07:11:51.992052+0000","last_fullsized":"2024-09-14T07:11:51.992052+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:43:51.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003558444,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8736,"num_read_kb":48195,"num_write":537,"num_write_kb":16927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1d"}]},{"pgid":"2.2","version":"19'2","reported_seq":440,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826271+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:11:49.826271+0000","last_peered":"2024-09-14T07:11:49.826271+0000","last_clean":"2024-09-14T07:11:49.826271+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:11:49.826271+0000","last_undegraded":"2024-09-14T07:11:49.826271+0000","last_fullsized":"2024-09-14T07:11:49.826271+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"217'500","reported_seq":6349,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993429+0000","last_change":"2024-09-14T07:11:49.823563+0000","last_active":"2024-09-14T07:11:49.993429+0000","last_peered":"2024-09-14T07:11:49.993429+0000","last_clean":"2024-09-14T07:11:49.993429+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:11:49.993429+0000","last_undegraded":"2024-09-14T07:11:49.993429+0000","last_fullsized":"2024-09-14T07:11:49.993429+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:33.400348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042663839999999998,"stat_sum":{"num_bytes":16,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":7616,"num_read_kb":23803,"num_write":401,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.1","version":"0'0","reported_seq":125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.817757+0000","last_change":"2024-09-14T07:10:52.793359+0000","last_active":"2024-09-14T07:11:49.817757+0000","last_peered":"2024-09-14T07:11:49.817757+0000","last_clean":"2024-09-14T07:11:49.817757+0000","last_became_active":"2024-09-14T07:10:52.792887+0000","last_became_peered":"2024-09-14T07:10:52.792887+0000","last_unstale":"2024-09-14T07:11:49.817757+0000","last_undegraded":"2024-09-14T07:11:49.817757+0000","last_fullsized":"2024-09-14T07:11:49.817757+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:11.029506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":12984,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819542+0000","last_change":"2024-09-14T07:11:49.822587+0000","last_active":"2024-09-14T07:11:50.819542+0000","last_peered":"2024-09-14T07:11:50.819542+0000","last_clean":"2024-09-14T07:11:50.819542+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:11:50.819542+0000","last_undegraded":"2024-09-14T07:11:50.819542+0000","last_fullsized":"2024-09-14T07:11:50.819542+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:12:01.776651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001171233,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.0","version":"0'0","reported_seq":430,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822692+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:11:49.822692+0000","last_peered":"2024-09-14T07:11:49.822692+0000","last_clean":"2024-09-14T07:11:49.822692+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:11:49.822692+0000","last_undegraded":"2024-09-14T07:11:49.822692+0000","last_fullsized":"2024-09-14T07:11:49.822692+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"213'396","reported_seq":7828,"reported_epoch":216,"state":"peering","last_fresh":"2024-09-14T07:11:48.983941+0000","last_change":"2024-09-14T07:11:49.818862+0000","last_active":"2024-09-14T07:11:49.818862+0000","last_peered":"2024-09-14T07:11:48.983941+0000","last_clean":"2024-09-14T07:11:46.103525+0000","last_became_active":"2024-09-14T07:11:48.811974+0000","last_became_peered":"2024-09-14T07:11:48.811974+0000","last_unstale":"2024-09-14T07:11:48.983941+0000","last_undegraded":"2024-09-14T07:11:48.983941+0000","last_fullsized":"2024-09-14T07:11:48.794907+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01901893,"stat_sum":{"num_bytes":858640,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":6436,"num_read_kb":16010,"num_write":342,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"2.3","version":"17'1","reported_seq":439,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.825168+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:11:49.825168+0000","last_peered":"2024-09-14T07:11:49.825168+0000","last_clean":"2024-09-14T07:11:49.825168+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:11:49.825168+0000","last_undegraded":"2024-09-14T07:11:49.825168+0000","last_fullsized":"2024-09-14T07:11:49.825168+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":542,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.822949+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:11:49.822949+0000","last_peered":"2024-09-14T07:11:49.822949+0000","last_clean":"2024-09-14T07:11:49.822949+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:11:49.822949+0000","last_undegraded":"2024-09-14T07:11:49.822949+0000","last_fullsized":"2024-09-14T07:11:49.822949+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"217'535","reported_seq":10138,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.820793+0000","last_change":"2024-09-14T07:11:49.820793+0000","last_active":"2024-09-14T07:11:49.820793+0000","last_peered":"2024-09-14T07:11:49.820793+0000","last_clean":"2024-09-14T07:11:49.820793+0000","last_became_active":"2024-09-14T07:10:58.788140+0000","last_became_peered":"2024-09-14T07:10:58.788140+0000","last_unstale":"2024-09-14T07:11:49.820793+0000","last_undegraded":"2024-09-14T07:11:49.820793+0000","last_fullsized":"2024-09-14T07:11:49.820793+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:21:00.558281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002853487,"stat_sum":{"num_bytes":7600655,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":7,"num_read":9538,"num_read_kb":69033,"num_write":685,"num_write_kb":27473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.9","version":"219'744","reported_seq":8360,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-14T07:11:52.030179+0000","last_change":"2024-09-14T07:11:51.842414+0000","last_active":"2024-09-14T07:11:52.030179+0000","last_peered":"2024-09-14T07:11:52.030179+0000","last_clean":"2024-09-14T07:11:52.030179+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:11:52.030179+0000","last_undegraded":"2024-09-14T07:11:52.030179+0000","last_fullsized":"2024-09-14T07:11:52.030179+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:58.263745+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051599389999999997,"stat_sum":{"num_bytes":76849,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":9,"num_read":10165,"num_read_kb":48111,"num_write":752,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1d"}]},{"pgid":"3.a","version":"217'437","reported_seq":8196,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.835504+0000","last_change":"2024-09-14T07:11:49.835504+0000","last_active":"2024-09-14T07:11:49.835504+0000","last_peered":"2024-09-14T07:11:49.835504+0000","last_clean":"2024-09-14T07:11:49.835504+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:11:49.835504+0000","last_undegraded":"2024-09-14T07:11:49.835504+0000","last_fullsized":"2024-09-14T07:11:49.835504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:24.697088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047041990000000001,"stat_sum":{"num_bytes":1452032,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":7826,"num_read_kb":35943,"num_write":413,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.b","version":"217'1019","reported_seq":8107,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.985806+0000","last_change":"2024-09-14T07:11:49.821839+0000","last_active":"2024-09-14T07:11:49.985806+0000","last_peered":"2024-09-14T07:11:49.985806+0000","last_clean":"2024-09-14T07:11:49.985806+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:11:49.985806+0000","last_undegraded":"2024-09-14T07:11:49.985806+0000","last_fullsized":"2024-09-14T07:11:49.985806+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:20:17.413757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036922539999999998,"stat_sum":{"num_bytes":393384,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":10228,"num_read_kb":41579,"num_write":1377,"num_write_kb":13967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.c","version":"217'547","reported_seq":15456,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830180+0000","last_change":"2024-09-14T07:11:49.830180+0000","last_active":"2024-09-14T07:11:49.830180+0000","last_peered":"2024-09-14T07:11:49.830180+0000","last_clean":"2024-09-14T07:11:49.830180+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:49.830180+0000","last_undegraded":"2024-09-14T07:11:49.830180+0000","last_fullsized":"2024-09-14T07:11:49.830180+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:03.498897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039997849999999996,"stat_sum":{"num_bytes":262144,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":463,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.d","version":"217'342","reported_seq":4125,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.993312+0000","last_change":"2024-09-14T07:11:49.834171+0000","last_active":"2024-09-14T07:11:49.993312+0000","last_peered":"2024-09-14T07:11:49.993312+0000","last_clean":"2024-09-14T07:11:49.993312+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:11:49.993312+0000","last_undegraded":"2024-09-14T07:11:49.993312+0000","last_fullsized":"2024-09-14T07:11:49.993312+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:13.447198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0071541449999999998,"stat_sum":{"num_bytes":1048576,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":323,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.e","version":"216'443","reported_seq":17414,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.304818+0000","last_change":"2024-09-14T07:11:50.307914+0000","last_active":"2024-09-14T07:11:50.307914+0000","last_peered":"2024-09-14T07:11:49.304818+0000","last_clean":"2024-09-14T07:11:40.083699+0000","last_became_active":"2024-09-14T07:11:48.814324+0000","last_became_peered":"2024-09-14T07:11:48.814324+0000","last_unstale":"2024-09-14T07:11:49.304818+0000","last_undegraded":"2024-09-14T07:11:49.304818+0000","last_fullsized":"2024-09-14T07:11:49.304818+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0016376439999999999,"stat_sum":{"num_bytes":524288,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":438,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.f","version":"217'484","reported_seq":8518,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.830984+0000","last_change":"2024-09-14T07:11:49.830984+0000","last_active":"2024-09-14T07:11:49.830984+0000","last_peered":"2024-09-14T07:11:49.830984+0000","last_clean":"2024-09-14T07:11:49.830984+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:11:49.830984+0000","last_undegraded":"2024-09-14T07:11:49.830984+0000","last_fullsized":"2024-09-14T07:11:49.830984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:49:26.120361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004728945,"stat_sum":{"num_bytes":131072,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":615,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.10","version":"216'462","reported_seq":8117,"reported_epoch":217,"state":"creating+peering","last_fresh":"2024-09-14T07:11:49.138923+0000","last_change":"2024-09-14T07:11:50.307757+0000","last_active":"2024-09-14T07:11:50.307757+0000","last_peered":"2024-09-14T07:11:49.138923+0000","last_clean":"2024-09-14T07:11:45.797283+0000","last_became_active":"2024-09-14T07:11:48.810645+0000","last_became_peered":"2024-09-14T07:11:48.810645+0000","last_unstale":"2024-09-14T07:11:49.138923+0000","last_undegraded":"2024-09-14T07:11:49.138923+0000","last_fullsized":"2024-09-14T07:11:48.791074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023065630000000002,"stat_sum":{"num_bytes":7864320,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":651,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"a","length":"1a"}]},{"pgid":"3.11","version":"217'537","reported_seq":9421,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.826638+0000","last_change":"2024-09-14T07:11:49.826638+0000","last_active":"2024-09-14T07:11:49.826638+0000","last_peered":"2024-09-14T07:11:49.826638+0000","last_clean":"2024-09-14T07:11:49.826638+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:11:49.826638+0000","last_undegraded":"2024-09-14T07:11:49.826638+0000","last_fullsized":"2024-09-14T07:11:49.826638+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:22:42.991862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036882769999999998,"stat_sum":{"num_bytes":17920512,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":10491,"num_read_kb":24428,"num_write":419,"num_write_kb":9181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.12","version":"217'558","reported_seq":7288,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.997627+0000","last_change":"2024-09-14T07:11:49.835043+0000","last_active":"2024-09-14T07:11:49.997627+0000","last_peered":"2024-09-14T07:11:49.997627+0000","last_clean":"2024-09-14T07:11:49.997627+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:11:49.997627+0000","last_undegraded":"2024-09-14T07:11:49.997627+0000","last_fullsized":"2024-09-14T07:11:49.997627+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:25:26.200557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005177877,"stat_sum":{"num_bytes":18746368,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":428,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.13","version":"217'381","reported_seq":4946,"reported_epoch":217,"state":"active+clean","last_fresh":"2024-09-14T07:11:49.831753+0000","last_change":"2024-09-14T07:11:49.831753+0000","last_active":"2024-09-14T07:11:49.831753+0000","last_peered":"2024-09-14T07:11:49.831753+0000","last_clean":"2024-09-14T07:11:49.831753+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:11:49.831753+0000","last_undegraded":"2024-09-14T07:11:49.831753+0000","last_fullsized":"2024-09-14T07:11:49.831753+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:29.134495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073722479999999996,"stat_sum":{"num_bytes":9211904,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":356,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"1c"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":126849862,"num_objects":335,"num_object_clones":9,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":254,"num_read":340606,"num_read_kb":1115608,"num_write":21848,"num_write_kb":329853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2318336,"data_stored":12842705,"data_compressed":1275026,"data_compressed_allocated":1765376,"data_compressed_original":12599296,"omap_allocated":12989,"internal_metadata":0},"log_size":19227,"ondisk_log_size":19227,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78448,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293392,"statfs":{"total":96636764160,"available":96556433408,"internally_reserved":0,"allocated":2908160,"data_stored":2763776,"data_compressed":139452,"data_compressed_allocated":212992,"data_compressed_original":1646592,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607608,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52124,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319716,"statfs":{"total":96636764160,"available":96583389184,"internally_reserved":0,"allocated":2957312,"data_stored":3660859,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607608,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70536,"kb_used_data":3196,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301304,"statfs":{"total":96636764160,"available":96564535296,"internally_reserved":0,"allocated":3272704,"data_stored":4258061,"data_compressed":351328,"data_compressed_allocated":512000,"data_compressed_original":3088384,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607609,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64988,"kb_used_data":3252,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306852,"statfs":{"total":96636764160,"available":96570216448,"internally_reserved":0,"allocated":3330048,"data_stored":6111343,"data_compressed":270417,"data_compressed_allocated":561152,"data_compressed_original":4960256,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607609,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48052,"kb_used_data":2980,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323788,"statfs":{"total":96636764160,"available":96587558912,"internally_reserved":0,"allocated":3051520,"data_stored":4216320,"data_compressed":171212,"data_compressed_allocated":348160,"data_compressed_original":3092480,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":2948,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":3018752,"data_stored":4267864,"data_compressed":256234,"data_compressed_allocated":327680,"data_compressed_original":3162112,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":2587034,"data_compressed":249342,"data_compressed_allocated":290816,"data_compressed_original":2572288,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2034009,"data_compressed":161836,"data_compressed_allocated":258048,"data_compressed_original":2002944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":1396187,"data_compressed":231909,"data_compressed_allocated":282624,"data_compressed_original":1343488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":2582245,"data_compressed":345270,"data_compressed_allocated":475136,"data_compressed_original":2514944,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":1667838,"data_compressed":139452,"data_compressed_allocated":212992,"data_compressed_original":1646592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":2575392,"data_compressed":147217,"data_compressed_allocated":245760,"data_compressed_original":2519040,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:11:55.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T07:11:55.771 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:55.771 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:55.785 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:11:55.828 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:55.825+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:55.891 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-09-14T07:11:55.885+0000 7f9bc306f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 40847) UID: 0 2024-09-14T07:11:55.965 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:55.965 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:55.981 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:11:56.032 INFO:tasks.ceph.osd.3.smithi110.stderr:2024-09-14T07:11:56.029+0000 7fa12f337640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25262) UID: 0 2024-09-14T07:11:56.034 INFO:teuthology.orchestra.run.smithi145.stdout:All operations completed A-OK! 2024-09-14T07:11:56.092 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-14T07:11:56.089+0000 7f8d20cdc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25263) UID: 0 2024-09-14T07:11:56.107 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:56.107 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:56.113 DEBUG:teuthology.parallel:result is None 2024-09-14T07:11:56.113 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T07:11:56.123 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T07:11:56.141 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T07:11:56.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:11:56.155 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:56.155 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":278,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":223,"num_osds":6,"num_up_osds":6,"osd_up_since":1726297902,"num_in_osds":6,"osd_in_since":1726297852,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"creating+peering","count":3},{"state_name":"peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":339,"data_bytes":127440249,"bytes_used":358760448,"bytes_avail":579461824512,"bytes_total":579820584960,"inactive_pgs_ratio":0.097560971975326538,"recovering_objects_per_sec":99,"recovering_bytes_per_sec":280874,"recovering_keys_per_sec":0,"num_objects_recovered":398,"num_bytes_recovered":1123856,"num_keys_recovered":0,"read_bytes_sec":127447,"write_bytes_sec":7933,"read_op_per_sec":145,"write_op_per_sec":38},"fsmap":{"epoch":1,"btime":"2024-09-14T07:07:18:019388+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T07:08:33.220112+0000","services":{}},"progress_events":{}} 2024-09-14T07:11:56.174 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T07:11:56.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:11:56.268 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:56.268 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:56.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:11:56.411 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:56.411 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:56.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:11:56.534 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:56.535 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:11:56.551 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-09-14T07:11:56.552 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:11:56.552 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-14T07:11:55.857385+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":55473575,"num_objects":285,"num_object_clones":2,"num_object_copies":570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":285,"num_whiteouts":250,"num_read":341394,"num_read_kb":1116278,"num_write":22088,"num_write_kb":331518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19463,"ondisk_log_size":19463,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":332744,"kb_used_data":17172,"kb_used_omap":35,"kb_used_meta":315356,"kb_avail":565898296,"statfs":{"total":579820584960,"available":579479855104,"internally_reserved":0,"allocated":17584128,"data_stored":14174222,"data_compressed":973506,"data_compressed_allocated":1310720,"data_compressed_original":7421952,"omap_allocated":36329,"internal_metadata":322925079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-73244297,"num_objects":-60,"num_object_clones":-8,"num_object_copies":-120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-60,"num_whiteouts":-5,"num_read":756,"num_read_kb":646,"num_write":150,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.027562"},"pg_stats":[{"pgid":"3.1f","version":"217'381","reported_seq":6950,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820211+0000","last_change":"2024-09-14T07:11:49.823797+0000","last_active":"2024-09-14T07:11:50.820211+0000","last_peered":"2024-09-14T07:11:50.820211+0000","last_clean":"2024-09-14T07:11:50.820211+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:11:50.820211+0000","last_undegraded":"2024-09-14T07:11:50.820211+0000","last_fullsized":"2024-09-14T07:11:50.820211+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:04:11.679482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081229,"stat_sum":{"num_bytes":16,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":12,"num_read":6401,"num_read_kb":50527,"num_write":555,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1e","version":"222'734","reported_seq":13021,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846599+0000","last_change":"2024-09-14T07:11:54.846599+0000","last_active":"2024-09-14T07:11:54.846599+0000","last_peered":"2024-09-14T07:11:54.846599+0000","last_clean":"2024-09-14T07:11:54.846599+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:11:54.846599+0000","last_undegraded":"2024-09-14T07:11:54.846599+0000","last_fullsized":"2024-09-14T07:11:54.846599+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:44:37.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024744020000000001,"stat_sum":{"num_bytes":503808,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":959,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.1d","version":"222'1612","reported_seq":16643,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.101494+0000","last_change":"2024-09-14T07:11:54.867528+0000","last_active":"2024-09-14T07:11:55.101494+0000","last_peered":"2024-09-14T07:11:55.101494+0000","last_clean":"2024-09-14T07:11:55.101494+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:11:55.101494+0000","last_undegraded":"2024-09-14T07:11:55.101494+0000","last_fullsized":"2024-09-14T07:11:55.101494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1612,"log_dups_size":0,"ondisk_log_size":1612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:43:57.674305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014680100000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":20960,"num_read_kb":36363,"num_write":1705,"num_write_kb":5681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.1c","version":"222'505","reported_seq":20839,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.916310+0000","last_change":"2024-09-14T07:11:54.867556+0000","last_active":"2024-09-14T07:11:54.916310+0000","last_peered":"2024-09-14T07:11:54.916310+0000","last_clean":"2024-09-14T07:11:54.916310+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:11:54.916310+0000","last_undegraded":"2024-09-14T07:11:54.916310+0000","last_fullsized":"2024-09-14T07:11:54.916310+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:44.787402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149807,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21052,"num_read_kb":46652,"num_write":509,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.1b","version":"214'572","reported_seq":8666,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819776+0000","last_change":"2024-09-14T07:11:49.823068+0000","last_active":"2024-09-14T07:11:50.819776+0000","last_peered":"2024-09-14T07:11:50.819776+0000","last_clean":"2024-09-14T07:11:50.819776+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:11:50.819776+0000","last_undegraded":"2024-09-14T07:11:50.819776+0000","last_fullsized":"2024-09-14T07:11:50.819776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:06:03.189849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015995510000000001,"stat_sum":{"num_bytes":19174036,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":8775,"num_read_kb":22847,"num_write":713,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.1a","version":"217'391","reported_seq":30324,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-14T07:11:52.008839+0000","last_change":"2024-09-14T07:11:51.838708+0000","last_active":"2024-09-14T07:11:52.008839+0000","last_peered":"2024-09-14T07:11:52.008839+0000","last_clean":"2024-09-14T07:11:52.008839+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:52.008839+0000","last_undegraded":"2024-09-14T07:11:52.008839+0000","last_fullsized":"2024-09-14T07:11:52.008839+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:48.578400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022117619999999999,"stat_sum":{"num_bytes":19,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":29657,"num_read_kb":50539,"num_write":415,"num_write_kb":12159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1d"}]},{"pgid":"3.19","version":"222'1352","reported_seq":10635,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.024790+0000","last_change":"2024-09-14T07:11:54.848906+0000","last_active":"2024-09-14T07:11:55.024790+0000","last_peered":"2024-09-14T07:11:55.024790+0000","last_clean":"2024-09-14T07:11:55.024790+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:11:55.024790+0000","last_undegraded":"2024-09-14T07:11:55.024790+0000","last_fullsized":"2024-09-14T07:11:55.024790+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1352,"log_dups_size":0,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:18:53.103645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002677806,"stat_sum":{"num_bytes":2576384,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2131,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.18","version":"218'427","reported_seq":5411,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.949888+0000","last_change":"2024-09-14T07:11:49.822391+0000","last_active":"2024-09-14T07:11:50.949888+0000","last_peered":"2024-09-14T07:11:50.949888+0000","last_clean":"2024-09-14T07:11:50.949888+0000","last_became_active":"2024-09-14T07:11:42.792094+0000","last_became_peered":"2024-09-14T07:11:42.792094+0000","last_unstale":"2024-09-14T07:11:50.949888+0000","last_undegraded":"2024-09-14T07:11:50.949888+0000","last_fullsized":"2024-09-14T07:11:50.949888+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:42:01.973449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096171600000000005,"stat_sum":{"num_bytes":270384,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":8,"num_read":4861,"num_read_kb":15257,"num_write":485,"num_write_kb":15533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"3.17","version":"217'242","reported_seq":3864,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.868270+0000","last_change":"2024-09-14T07:11:54.868664+0000","last_active":"2024-09-14T07:11:54.868270+0000","last_peered":"2024-09-14T07:11:54.868270+0000","last_clean":"2024-09-14T07:11:54.868270+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:11:54.868270+0000","last_undegraded":"2024-09-14T07:11:54.868270+0000","last_fullsized":"2024-09-14T07:11:54.868270+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:42.760188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023916789999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.16","version":"218'643","reported_seq":10046,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.867608+0000","last_change":"2024-09-14T07:11:54.868072+0000","last_active":"2024-09-14T07:11:54.867608+0000","last_peered":"2024-09-14T07:11:54.867608+0000","last_clean":"2024-09-14T07:11:54.867608+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:11:54.867608+0000","last_undegraded":"2024-09-14T07:11:54.867608+0000","last_fullsized":"2024-09-14T07:11:54.867608+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:05:49.150095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019758290000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.15","version":"222'537","reported_seq":7105,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.103083+0000","last_change":"2024-09-14T07:11:54.846025+0000","last_active":"2024-09-14T07:11:55.103083+0000","last_peered":"2024-09-14T07:11:55.103083+0000","last_clean":"2024-09-14T07:11:55.103083+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:11:55.103083+0000","last_undegraded":"2024-09-14T07:11:55.103083+0000","last_fullsized":"2024-09-14T07:11:55.103083+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:52:20.130015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002406638,"stat_sum":{"num_bytes":3223552,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":542,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.14","version":"222'756","reported_seq":11111,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.009986+0000","last_change":"2024-09-14T07:11:54.845548+0000","last_active":"2024-09-14T07:11:55.009986+0000","last_peered":"2024-09-14T07:11:55.009986+0000","last_clean":"2024-09-14T07:11:55.009986+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:11:55.009986+0000","last_undegraded":"2024-09-14T07:11:55.009986+0000","last_fullsized":"2024-09-14T07:11:55.009986+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:22:13.233639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019772029999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":440,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.844243+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:11:54.844243+0000","last_peered":"2024-09-14T07:11:54.844243+0000","last_clean":"2024-09-14T07:11:54.844243+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:11:54.844243+0000","last_undegraded":"2024-09-14T07:11:54.844243+0000","last_fullsized":"2024-09-14T07:11:54.844243+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"218'692","reported_seq":12786,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.820180+0000","last_change":"2024-09-14T07:11:49.823661+0000","last_active":"2024-09-14T07:11:50.820180+0000","last_peered":"2024-09-14T07:11:50.820180+0000","last_clean":"2024-09-14T07:11:50.820180+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:11:50.820180+0000","last_undegraded":"2024-09-14T07:11:50.820180+0000","last_fullsized":"2024-09-14T07:11:50.820180+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:52:36.850781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001773719,"stat_sum":{"num_bytes":129628,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":5,"num_read":14012,"num_read_kb":35141,"num_write":738,"num_write_kb":5113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846362+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:11:54.846362+0000","last_peered":"2024-09-14T07:11:54.846362+0000","last_clean":"2024-09-14T07:11:54.846362+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:11:54.846362+0000","last_undegraded":"2024-09-14T07:11:54.846362+0000","last_fullsized":"2024-09-14T07:11:54.846362+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"217'658","reported_seq":8347,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.850639+0000","last_change":"2024-09-14T07:11:54.851711+0000","last_active":"2024-09-14T07:11:54.850639+0000","last_peered":"2024-09-14T07:11:54.850639+0000","last_clean":"2024-09-14T07:11:54.850639+0000","last_became_active":"2024-09-14T07:10:57.225048+0000","last_became_peered":"2024-09-14T07:10:57.225048+0000","last_unstale":"2024-09-14T07:11:54.850639+0000","last_undegraded":"2024-09-14T07:11:54.850639+0000","last_fullsized":"2024-09-14T07:11:54.850639+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:57:25.686031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":701,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":449,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.843922+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:11:54.843922+0000","last_peered":"2024-09-14T07:11:54.843922+0000","last_clean":"2024-09-14T07:11:54.843922+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:11:54.843922+0000","last_undegraded":"2024-09-14T07:11:54.843922+0000","last_fullsized":"2024-09-14T07:11:54.843922+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6702,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.849202+0000","last_change":"2024-09-14T07:11:54.850813+0000","last_active":"2024-09-14T07:11:54.849202+0000","last_peered":"2024-09-14T07:11:54.849202+0000","last_clean":"2024-09-14T07:11:54.849202+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:11:54.849202+0000","last_undegraded":"2024-09-14T07:11:54.849202+0000","last_fullsized":"2024-09-14T07:11:54.849202+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:53:47.581853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044065470000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846264+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:11:54.846264+0000","last_peered":"2024-09-14T07:11:54.846264+0000","last_clean":"2024-09-14T07:11:54.846264+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:11:54.846264+0000","last_undegraded":"2024-09-14T07:11:54.846264+0000","last_fullsized":"2024-09-14T07:11:54.846264+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"221'754","reported_seq":4060,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.868931+0000","last_change":"2024-09-14T07:11:54.869235+0000","last_active":"2024-09-14T07:11:54.868931+0000","last_peered":"2024-09-14T07:11:54.868931+0000","last_clean":"2024-09-14T07:11:54.868931+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:11:54.868931+0000","last_undegraded":"2024-09-14T07:11:54.868931+0000","last_fullsized":"2024-09-14T07:11:54.868931+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:02:15.768862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002796088,"stat_sum":{"num_bytes":4629504,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1114,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.3","version":"219'498","reported_seq":10071,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-14T07:11:51.992052+0000","last_change":"2024-09-14T07:11:51.839726+0000","last_active":"2024-09-14T07:11:51.992052+0000","last_peered":"2024-09-14T07:11:51.992052+0000","last_clean":"2024-09-14T07:11:51.992052+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:11:51.992052+0000","last_undegraded":"2024-09-14T07:11:51.992052+0000","last_fullsized":"2024-09-14T07:11:51.992052+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:43:51.940874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003558444,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8736,"num_read_kb":48195,"num_write":537,"num_write_kb":16927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1d"}]},{"pgid":"2.2","version":"19'2","reported_seq":450,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866404+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:11:54.866404+0000","last_peered":"2024-09-14T07:11:54.866404+0000","last_clean":"2024-09-14T07:11:54.866404+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:11:54.866404+0000","last_undegraded":"2024-09-14T07:11:54.866404+0000","last_fullsized":"2024-09-14T07:11:54.866404+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6382,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.850932+0000","last_change":"2024-09-14T07:11:54.852420+0000","last_active":"2024-09-14T07:11:54.850932+0000","last_peered":"2024-09-14T07:11:54.850932+0000","last_clean":"2024-09-14T07:11:54.850932+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:11:54.850932+0000","last_undegraded":"2024-09-14T07:11:54.850932+0000","last_fullsized":"2024-09-14T07:11:54.850932+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:59:39.649601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055133680000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.845683+0000","last_change":"2024-09-14T07:10:52.793359+0000","last_active":"2024-09-14T07:11:54.845683+0000","last_peered":"2024-09-14T07:11:54.845683+0000","last_clean":"2024-09-14T07:11:54.845683+0000","last_became_active":"2024-09-14T07:10:52.792887+0000","last_became_peered":"2024-09-14T07:10:52.792887+0000","last_unstale":"2024-09-14T07:11:54.845683+0000","last_undegraded":"2024-09-14T07:11:54.845683+0000","last_fullsized":"2024-09-14T07:11:54.845683+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:11.029506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":12984,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-14T07:11:50.819542+0000","last_change":"2024-09-14T07:11:49.822587+0000","last_active":"2024-09-14T07:11:50.819542+0000","last_peered":"2024-09-14T07:11:50.819542+0000","last_clean":"2024-09-14T07:11:50.819542+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:11:50.819542+0000","last_undegraded":"2024-09-14T07:11:50.819542+0000","last_fullsized":"2024-09-14T07:11:50.819542+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:12:01.776651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001171233,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"1c"}]},{"pgid":"2.0","version":"0'0","reported_seq":440,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.843886+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:11:54.843886+0000","last_peered":"2024-09-14T07:11:54.843886+0000","last_clean":"2024-09-14T07:11:54.843886+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:11:54.843886+0000","last_undegraded":"2024-09-14T07:11:54.843886+0000","last_fullsized":"2024-09-14T07:11:54.843886+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7881,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.962295+0000","last_change":"2024-09-14T07:11:54.850454+0000","last_active":"2024-09-14T07:11:54.962295+0000","last_peered":"2024-09-14T07:11:54.962295+0000","last_clean":"2024-09-14T07:11:54.962295+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:11:54.962295+0000","last_undegraded":"2024-09-14T07:11:54.962295+0000","last_fullsized":"2024-09-14T07:11:54.962295+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:07:04.395046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004048472,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.3","version":"17'1","reported_seq":449,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866001+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:11:54.866001+0000","last_peered":"2024-09-14T07:11:54.866001+0000","last_clean":"2024-09-14T07:11:54.866001+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:11:54.866001+0000","last_undegraded":"2024-09-14T07:11:54.866001+0000","last_fullsized":"2024-09-14T07:11:54.866001+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":552,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.843345+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:11:54.843345+0000","last_peered":"2024-09-14T07:11:54.843345+0000","last_clean":"2024-09-14T07:11:54.843345+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:11:54.843345+0000","last_undegraded":"2024-09-14T07:11:54.843345+0000","last_fullsized":"2024-09-14T07:11:54.843345+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"222'562","reported_seq":10278,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.848895+0000","last_change":"2024-09-14T07:11:54.848895+0000","last_active":"2024-09-14T07:11:54.848895+0000","last_peered":"2024-09-14T07:11:54.848895+0000","last_clean":"2024-09-14T07:11:54.848895+0000","last_became_active":"2024-09-14T07:10:58.788140+0000","last_became_peered":"2024-09-14T07:10:58.788140+0000","last_unstale":"2024-09-14T07:11:54.848895+0000","last_undegraded":"2024-09-14T07:11:54.848895+0000","last_fullsized":"2024-09-14T07:11:54.848895+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:24:40.252737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":4106752,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":705,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.9","version":"219'744","reported_seq":8360,"reported_epoch":219,"state":"active+clean","last_fresh":"2024-09-14T07:11:52.030179+0000","last_change":"2024-09-14T07:11:51.842414+0000","last_active":"2024-09-14T07:11:52.030179+0000","last_peered":"2024-09-14T07:11:52.030179+0000","last_clean":"2024-09-14T07:11:52.030179+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:11:52.030179+0000","last_undegraded":"2024-09-14T07:11:52.030179+0000","last_fullsized":"2024-09-14T07:11:52.030179+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:58.263745+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051599389999999997,"stat_sum":{"num_bytes":76849,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":9,"num_read":10165,"num_read_kb":48111,"num_write":752,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"7","length":"1d"}]},{"pgid":"3.a","version":"221'439","reported_seq":8228,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.845019+0000","last_change":"2024-09-14T07:11:54.845576+0000","last_active":"2024-09-14T07:11:54.845019+0000","last_peered":"2024-09-14T07:11:54.845019+0000","last_clean":"2024-09-14T07:11:54.845019+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:11:54.845019+0000","last_undegraded":"2024-09-14T07:11:54.845019+0000","last_fullsized":"2024-09-14T07:11:54.845019+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:01.479158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020058710000000002,"stat_sum":{"num_bytes":1452032,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":415,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.b","version":"221'1024","reported_seq":8144,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.852373+0000","last_change":"2024-09-14T07:11:54.853379+0000","last_active":"2024-09-14T07:11:54.852373+0000","last_peered":"2024-09-14T07:11:54.852373+0000","last_clean":"2024-09-14T07:11:54.852373+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:11:54.852373+0000","last_undegraded":"2024-09-14T07:11:54.852373+0000","last_fullsized":"2024-09-14T07:11:54.852373+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:51:36.820115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056048269999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.c","version":"220'550","reported_seq":15489,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.868148+0000","last_change":"2024-09-14T07:11:54.868605+0000","last_active":"2024-09-14T07:11:54.868148+0000","last_peered":"2024-09-14T07:11:54.868148+0000","last_clean":"2024-09-14T07:11:54.868148+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:54.868148+0000","last_undegraded":"2024-09-14T07:11:54.868148+0000","last_fullsized":"2024-09-14T07:11:54.868148+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:34:41.751207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351639,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.d","version":"222'344","reported_seq":4157,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.015260+0000","last_change":"2024-09-14T07:11:54.846341+0000","last_active":"2024-09-14T07:11:55.015260+0000","last_peered":"2024-09-14T07:11:55.015260+0000","last_clean":"2024-09-14T07:11:55.015260+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:11:55.015260+0000","last_undegraded":"2024-09-14T07:11:55.015260+0000","last_fullsized":"2024-09-14T07:11:55.015260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:50.694221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023414690000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.e","version":"220'446","reported_seq":17462,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.867671+0000","last_change":"2024-09-14T07:11:54.868113+0000","last_active":"2024-09-14T07:11:54.867671+0000","last_peered":"2024-09-14T07:11:54.867671+0000","last_clean":"2024-09-14T07:11:54.867671+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:11:54.867671+0000","last_undegraded":"2024-09-14T07:11:54.867671+0000","last_fullsized":"2024-09-14T07:11:54.867671+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:54:55.683061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001943455,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.f","version":"221'485","reported_seq":8549,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866400+0000","last_change":"2024-09-14T07:11:54.866888+0000","last_active":"2024-09-14T07:11:54.866400+0000","last_peered":"2024-09-14T07:11:54.866400+0000","last_clean":"2024-09-14T07:11:54.866400+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:11:54.866400+0000","last_undegraded":"2024-09-14T07:11:54.866400+0000","last_fullsized":"2024-09-14T07:11:54.866400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:57:48.062313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086905399999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.10","version":"222'463","reported_seq":8163,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866954+0000","last_change":"2024-09-14T07:11:54.866954+0000","last_active":"2024-09-14T07:11:54.866954+0000","last_peered":"2024-09-14T07:11:54.866954+0000","last_clean":"2024-09-14T07:11:54.866954+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:11:54.866954+0000","last_undegraded":"2024-09-14T07:11:54.866954+0000","last_fullsized":"2024-09-14T07:11:54.866954+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:10:12.283798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010057550000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.11","version":"222'560","reported_seq":9546,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846947+0000","last_change":"2024-09-14T07:11:54.846947+0000","last_active":"2024-09-14T07:11:54.846947+0000","last_peered":"2024-09-14T07:11:54.846947+0000","last_clean":"2024-09-14T07:11:54.846947+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:11:54.846947+0000","last_undegraded":"2024-09-14T07:11:54.846947+0000","last_fullsized":"2024-09-14T07:11:54.846947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:29:32.956389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002649775,"stat_sum":{"num_bytes":3663872,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":437,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.12","version":"222'559","reported_seq":7319,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.844940+0000","last_change":"2024-09-14T07:11:54.844940+0000","last_active":"2024-09-14T07:11:54.844940+0000","last_peered":"2024-09-14T07:11:54.844940+0000","last_clean":"2024-09-14T07:11:54.844940+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:11:54.844940+0000","last_undegraded":"2024-09-14T07:11:54.844940+0000","last_fullsized":"2024-09-14T07:11:54.844940+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:38:40.357896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001463347,"stat_sum":{"num_bytes":10882048,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":429,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.13","version":"222'383","reported_seq":4978,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.844796+0000","last_change":"2024-09-14T07:11:54.844796+0000","last_active":"2024-09-14T07:11:54.844796+0000","last_peered":"2024-09-14T07:11:54.844796+0000","last_clean":"2024-09-14T07:11:54.844796+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:11:54.844796+0000","last_undegraded":"2024-09-14T07:11:54.844796+0000","last_fullsized":"2024-09-14T07:11:54.844796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:55:43.278292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001280376,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":54883188,"num_objects":281,"num_object_clones":2,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":250,"num_read":341328,"num_read_kb":1116223,"num_write":21985,"num_write_kb":329888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1056768,"data_stored":5430489,"data_compressed":622105,"data_compressed_allocated":827392,"data_compressed_original":5349376,"omap_allocated":12989,"internal_metadata":0},"log_size":19388,"ondisk_log_size":19388,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78448,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293392,"statfs":{"total":96636764160,"available":96556433408,"internally_reserved":0,"allocated":2908160,"data_stored":2763776,"data_compressed":139452,"data_compressed_allocated":212992,"data_compressed_original":1646592,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607609,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51924,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319916,"statfs":{"total":96636764160,"available":96583593984,"internally_reserved":0,"allocated":2744320,"data_stored":1300489,"data_compressed":32306,"data_compressed_allocated":45056,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607609,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70360,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301480,"statfs":{"total":96636764160,"available":96564715520,"internally_reserved":0,"allocated":3141632,"data_stored":2593879,"data_compressed":273623,"data_compressed_allocated":376832,"data_compressed_original":1433600,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607610,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47956,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323884,"statfs":{"total":96636764160,"available":96587657216,"internally_reserved":0,"allocated":2940928,"data_stored":1816863,"data_compressed":159177,"data_compressed_allocated":212992,"data_compressed_original":684032,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607610,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47852,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323988,"statfs":{"total":96636764160,"available":96587763712,"internally_reserved":0,"allocated":2830336,"data_stored":1431351,"data_compressed":112714,"data_compressed_allocated":135168,"data_compressed_original":315392,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":2948,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":3018752,"data_stored":4267864,"data_compressed":256234,"data_compressed_allocated":327680,"data_compressed_original":3162112,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":2587034,"data_compressed":249342,"data_compressed_allocated":290816,"data_compressed_original":2572288,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":209920,"data_compressed":46844,"data_compressed_allocated":61440,"data_compressed_original":200704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":406577,"data_compressed":68930,"data_compressed_allocated":98304,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":370704,"data_compressed":85231,"data_compressed_allocated":118784,"data_compressed_original":360448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":1667838,"data_compressed":139452,"data_compressed_allocated":212992,"data_compressed_original":1646592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":188416,"data_compressed":32306,"data_compressed_allocated":45056,"data_compressed_original":180224,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:11:56.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:11:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-09-14T07:11:57.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:11:58.874 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-09-14T07:11:59.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:11:59.881 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:11:59.882 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:11:59.900 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-14T07:11:59.273044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":35822662,"num_objects":270,"num_object_clones":0,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":249,"num_read":341741,"num_read_kb":1116576,"num_write":22169,"num_write_kb":331550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19552,"ondisk_log_size":19552,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336420,"kb_used_data":16680,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894620,"statfs":{"total":579820584960,"available":579476090880,"internally_reserved":0,"allocated":17080320,"data_stored":9967864,"data_compressed":584712,"data_compressed_allocated":806912,"data_compressed_original":3203072,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-91617587,"num_objects":-69,"num_object_clones":-9,"num_object_copies":-138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-69,"num_whiteouts":-5,"num_read":1069,"num_read_kb":913,"num_write":218,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.431337"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":6988,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.907742+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:11:55.907742+0000","last_peered":"2024-09-14T07:11:55.907742+0000","last_clean":"2024-09-14T07:11:55.907742+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:11:55.907742+0000","last_undegraded":"2024-09-14T07:11:55.907742+0000","last_fullsized":"2024-09-14T07:11:55.907742+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1e","version":"222'734","reported_seq":13021,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846599+0000","last_change":"2024-09-14T07:11:54.846599+0000","last_active":"2024-09-14T07:11:54.846599+0000","last_peered":"2024-09-14T07:11:54.846599+0000","last_clean":"2024-09-14T07:11:54.846599+0000","last_became_active":"2024-09-14T07:10:12.820038+0000","last_became_peered":"2024-09-14T07:10:12.820038+0000","last_unstale":"2024-09-14T07:11:54.846599+0000","last_undegraded":"2024-09-14T07:11:54.846599+0000","last_fullsized":"2024-09-14T07:11:54.846599+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:44:37.524890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024744020000000001,"stat_sum":{"num_bytes":503808,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":959,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.1d","version":"222'1612","reported_seq":16643,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.101494+0000","last_change":"2024-09-14T07:11:54.867528+0000","last_active":"2024-09-14T07:11:55.101494+0000","last_peered":"2024-09-14T07:11:55.101494+0000","last_clean":"2024-09-14T07:11:55.101494+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:11:55.101494+0000","last_undegraded":"2024-09-14T07:11:55.101494+0000","last_fullsized":"2024-09-14T07:11:55.101494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1612,"log_dups_size":0,"ondisk_log_size":1612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:43:57.674305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014680100000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":20960,"num_read_kb":36363,"num_write":1705,"num_write_kb":5681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.1c","version":"222'505","reported_seq":20839,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.916310+0000","last_change":"2024-09-14T07:11:54.867556+0000","last_active":"2024-09-14T07:11:54.916310+0000","last_peered":"2024-09-14T07:11:54.916310+0000","last_clean":"2024-09-14T07:11:54.916310+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:11:54.916310+0000","last_undegraded":"2024-09-14T07:11:54.916310+0000","last_fullsized":"2024-09-14T07:11:54.916310+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:01:44.787402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00149807,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21052,"num_read_kb":46652,"num_write":509,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.1b","version":"222'576","reported_seq":8706,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.905580+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:11:55.905580+0000","last_peered":"2024-09-14T07:11:55.905580+0000","last_clean":"2024-09-14T07:11:55.905580+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:11:55.905580+0000","last_undegraded":"2024-09-14T07:11:55.905580+0000","last_fullsized":"2024-09-14T07:11:55.905580+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.1a","version":"222'413","reported_seq":30478,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.926383+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:11:55.926383+0000","last_peered":"2024-09-14T07:11:55.926383+0000","last_clean":"2024-09-14T07:11:55.926383+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:55.926383+0000","last_undegraded":"2024-09-14T07:11:55.926383+0000","last_fullsized":"2024-09-14T07:11:55.926383+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.19","version":"222'1352","reported_seq":10635,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.024790+0000","last_change":"2024-09-14T07:11:54.848906+0000","last_active":"2024-09-14T07:11:55.024790+0000","last_peered":"2024-09-14T07:11:55.024790+0000","last_clean":"2024-09-14T07:11:55.024790+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:11:55.024790+0000","last_undegraded":"2024-09-14T07:11:55.024790+0000","last_fullsized":"2024-09-14T07:11:55.024790+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1352,"log_dups_size":0,"ondisk_log_size":1352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:18:53.103645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002677806,"stat_sum":{"num_bytes":2576384,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2131,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.18","version":"223'441","reported_seq":5463,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.906145+0000","last_change":"2024-09-14T07:11:55.906145+0000","last_active":"2024-09-14T07:11:55.906145+0000","last_peered":"2024-09-14T07:11:55.906145+0000","last_clean":"2024-09-14T07:11:55.906145+0000","last_became_active":"2024-09-14T07:11:42.792094+0000","last_became_peered":"2024-09-14T07:11:42.792094+0000","last_unstale":"2024-09-14T07:11:55.906145+0000","last_undegraded":"2024-09-14T07:11:55.906145+0000","last_fullsized":"2024-09-14T07:11:55.906145+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:23:35.591685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.17","version":"217'242","reported_seq":3864,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.868270+0000","last_change":"2024-09-14T07:11:54.868664+0000","last_active":"2024-09-14T07:11:54.868270+0000","last_peered":"2024-09-14T07:11:54.868270+0000","last_clean":"2024-09-14T07:11:54.868270+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:11:54.868270+0000","last_undegraded":"2024-09-14T07:11:54.868270+0000","last_fullsized":"2024-09-14T07:11:54.868270+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:42.760188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023916789999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.16","version":"218'643","reported_seq":10046,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.867608+0000","last_change":"2024-09-14T07:11:54.868072+0000","last_active":"2024-09-14T07:11:54.867608+0000","last_peered":"2024-09-14T07:11:54.867608+0000","last_clean":"2024-09-14T07:11:54.867608+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:11:54.867608+0000","last_undegraded":"2024-09-14T07:11:54.867608+0000","last_fullsized":"2024-09-14T07:11:54.867608+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:05:49.150095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019758290000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.15","version":"222'537","reported_seq":7105,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.103083+0000","last_change":"2024-09-14T07:11:54.846025+0000","last_active":"2024-09-14T07:11:55.103083+0000","last_peered":"2024-09-14T07:11:55.103083+0000","last_clean":"2024-09-14T07:11:55.103083+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:11:55.103083+0000","last_undegraded":"2024-09-14T07:11:55.103083+0000","last_fullsized":"2024-09-14T07:11:55.103083+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:52:20.130015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002406638,"stat_sum":{"num_bytes":3223552,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":542,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.14","version":"222'756","reported_seq":11111,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.009986+0000","last_change":"2024-09-14T07:11:54.845548+0000","last_active":"2024-09-14T07:11:55.009986+0000","last_peered":"2024-09-14T07:11:55.009986+0000","last_clean":"2024-09-14T07:11:55.009986+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:11:55.009986+0000","last_undegraded":"2024-09-14T07:11:55.009986+0000","last_fullsized":"2024-09-14T07:11:55.009986+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:22:13.233639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019772029999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":440,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.844243+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:11:54.844243+0000","last_peered":"2024-09-14T07:11:54.844243+0000","last_clean":"2024-09-14T07:11:54.844243+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:11:54.844243+0000","last_undegraded":"2024-09-14T07:11:54.844243+0000","last_fullsized":"2024-09-14T07:11:54.844243+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12844,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.907240+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:11:55.907240+0000","last_peered":"2024-09-14T07:11:55.907240+0000","last_clean":"2024-09-14T07:11:55.907240+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:11:55.907240+0000","last_undegraded":"2024-09-14T07:11:55.907240+0000","last_fullsized":"2024-09-14T07:11:55.907240+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846362+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:11:54.846362+0000","last_peered":"2024-09-14T07:11:54.846362+0000","last_clean":"2024-09-14T07:11:54.846362+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:11:54.846362+0000","last_undegraded":"2024-09-14T07:11:54.846362+0000","last_fullsized":"2024-09-14T07:11:54.846362+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"217'658","reported_seq":8347,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.850639+0000","last_change":"2024-09-14T07:11:54.851711+0000","last_active":"2024-09-14T07:11:54.850639+0000","last_peered":"2024-09-14T07:11:54.850639+0000","last_clean":"2024-09-14T07:11:54.850639+0000","last_became_active":"2024-09-14T07:10:57.225048+0000","last_became_peered":"2024-09-14T07:10:57.225048+0000","last_unstale":"2024-09-14T07:11:54.850639+0000","last_undegraded":"2024-09-14T07:11:54.850639+0000","last_fullsized":"2024-09-14T07:11:54.850639+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:57:25.686031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":701,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.5","version":"0'0","reported_seq":449,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.843922+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:11:54.843922+0000","last_peered":"2024-09-14T07:11:54.843922+0000","last_clean":"2024-09-14T07:11:54.843922+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:11:54.843922+0000","last_undegraded":"2024-09-14T07:11:54.843922+0000","last_fullsized":"2024-09-14T07:11:54.843922+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6702,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.849202+0000","last_change":"2024-09-14T07:11:54.850813+0000","last_active":"2024-09-14T07:11:54.849202+0000","last_peered":"2024-09-14T07:11:54.849202+0000","last_clean":"2024-09-14T07:11:54.849202+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:11:54.849202+0000","last_undegraded":"2024-09-14T07:11:54.849202+0000","last_fullsized":"2024-09-14T07:11:54.849202+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:53:47.581853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044065470000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846264+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:11:54.846264+0000","last_peered":"2024-09-14T07:11:54.846264+0000","last_clean":"2024-09-14T07:11:54.846264+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:11:54.846264+0000","last_undegraded":"2024-09-14T07:11:54.846264+0000","last_fullsized":"2024-09-14T07:11:54.846264+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"221'754","reported_seq":4060,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.868931+0000","last_change":"2024-09-14T07:11:54.869235+0000","last_active":"2024-09-14T07:11:54.868931+0000","last_peered":"2024-09-14T07:11:54.868931+0000","last_clean":"2024-09-14T07:11:54.868931+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:11:54.868931+0000","last_undegraded":"2024-09-14T07:11:54.868931+0000","last_fullsized":"2024-09-14T07:11:54.868931+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:02:15.768862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002796088,"stat_sum":{"num_bytes":4629504,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1114,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.3","version":"223'514","reported_seq":10131,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:56.033324+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:11:56.033324+0000","last_peered":"2024-09-14T07:11:56.033324+0000","last_clean":"2024-09-14T07:11:56.033324+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:11:56.033324+0000","last_undegraded":"2024-09-14T07:11:56.033324+0000","last_fullsized":"2024-09-14T07:11:56.033324+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.2","version":"19'2","reported_seq":450,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866404+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:11:54.866404+0000","last_peered":"2024-09-14T07:11:54.866404+0000","last_clean":"2024-09-14T07:11:54.866404+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:11:54.866404+0000","last_undegraded":"2024-09-14T07:11:54.866404+0000","last_fullsized":"2024-09-14T07:11:54.866404+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6382,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.850932+0000","last_change":"2024-09-14T07:11:54.852420+0000","last_active":"2024-09-14T07:11:54.850932+0000","last_peered":"2024-09-14T07:11:54.850932+0000","last_clean":"2024-09-14T07:11:54.850932+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:11:54.850932+0000","last_undegraded":"2024-09-14T07:11:54.850932+0000","last_fullsized":"2024-09-14T07:11:54.850932+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:59:39.649601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0055133680000000003,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":135,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.845683+0000","last_change":"2024-09-14T07:10:52.793359+0000","last_active":"2024-09-14T07:11:54.845683+0000","last_peered":"2024-09-14T07:11:54.845683+0000","last_clean":"2024-09-14T07:11:54.845683+0000","last_became_active":"2024-09-14T07:10:52.792887+0000","last_became_peered":"2024-09-14T07:10:52.792887+0000","last_unstale":"2024-09-14T07:11:54.845683+0000","last_undegraded":"2024-09-14T07:11:54.845683+0000","last_fullsized":"2024-09-14T07:11:54.845683+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:16:11.029506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13019,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.906208+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:11:55.906208+0000","last_peered":"2024-09-14T07:11:55.906208+0000","last_clean":"2024-09-14T07:11:55.906208+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:11:55.906208+0000","last_undegraded":"2024-09-14T07:11:55.906208+0000","last_fullsized":"2024-09-14T07:11:55.906208+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":440,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.843886+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:11:54.843886+0000","last_peered":"2024-09-14T07:11:54.843886+0000","last_clean":"2024-09-14T07:11:54.843886+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:11:54.843886+0000","last_undegraded":"2024-09-14T07:11:54.843886+0000","last_fullsized":"2024-09-14T07:11:54.843886+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7881,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.962295+0000","last_change":"2024-09-14T07:11:54.850454+0000","last_active":"2024-09-14T07:11:54.962295+0000","last_peered":"2024-09-14T07:11:54.962295+0000","last_clean":"2024-09-14T07:11:54.962295+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:11:54.962295+0000","last_undegraded":"2024-09-14T07:11:54.962295+0000","last_fullsized":"2024-09-14T07:11:54.962295+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:07:04.395046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004048472,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"2.3","version":"17'1","reported_seq":449,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866001+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:11:54.866001+0000","last_peered":"2024-09-14T07:11:54.866001+0000","last_clean":"2024-09-14T07:11:54.866001+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:11:54.866001+0000","last_undegraded":"2024-09-14T07:11:54.866001+0000","last_fullsized":"2024-09-14T07:11:54.866001+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":552,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.843345+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:11:54.843345+0000","last_peered":"2024-09-14T07:11:54.843345+0000","last_clean":"2024-09-14T07:11:54.843345+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:11:54.843345+0000","last_undegraded":"2024-09-14T07:11:54.843345+0000","last_fullsized":"2024-09-14T07:11:54.843345+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"222'562","reported_seq":10278,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.848895+0000","last_change":"2024-09-14T07:11:54.848895+0000","last_active":"2024-09-14T07:11:54.848895+0000","last_peered":"2024-09-14T07:11:54.848895+0000","last_clean":"2024-09-14T07:11:54.848895+0000","last_became_active":"2024-09-14T07:10:58.788140+0000","last_became_peered":"2024-09-14T07:10:58.788140+0000","last_unstale":"2024-09-14T07:11:54.848895+0000","last_undegraded":"2024-09-14T07:11:54.848895+0000","last_fullsized":"2024-09-14T07:11:54.848895+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:24:40.252737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":4106752,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":705,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.9","version":"223'757","reported_seq":8405,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-14T07:11:56.014973+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:11:56.014973+0000","last_peered":"2024-09-14T07:11:56.014973+0000","last_clean":"2024-09-14T07:11:56.014973+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:11:56.014973+0000","last_undegraded":"2024-09-14T07:11:56.014973+0000","last_fullsized":"2024-09-14T07:11:56.014973+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"7"}]},{"pgid":"3.a","version":"221'439","reported_seq":8228,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.845019+0000","last_change":"2024-09-14T07:11:54.845576+0000","last_active":"2024-09-14T07:11:54.845019+0000","last_peered":"2024-09-14T07:11:54.845019+0000","last_clean":"2024-09-14T07:11:54.845019+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:11:54.845019+0000","last_undegraded":"2024-09-14T07:11:54.845019+0000","last_fullsized":"2024-09-14T07:11:54.845019+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:01.479158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020058710000000002,"stat_sum":{"num_bytes":1452032,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":415,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.b","version":"221'1024","reported_seq":8144,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.852373+0000","last_change":"2024-09-14T07:11:54.853379+0000","last_active":"2024-09-14T07:11:54.852373+0000","last_peered":"2024-09-14T07:11:54.852373+0000","last_clean":"2024-09-14T07:11:54.852373+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:11:54.852373+0000","last_undegraded":"2024-09-14T07:11:54.852373+0000","last_fullsized":"2024-09-14T07:11:54.852373+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:51:36.820115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0056048269999999997,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.c","version":"220'550","reported_seq":15489,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.868148+0000","last_change":"2024-09-14T07:11:54.868605+0000","last_active":"2024-09-14T07:11:54.868148+0000","last_peered":"2024-09-14T07:11:54.868148+0000","last_clean":"2024-09-14T07:11:54.868148+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:11:54.868148+0000","last_undegraded":"2024-09-14T07:11:54.868148+0000","last_fullsized":"2024-09-14T07:11:54.868148+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:34:41.751207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002351639,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.d","version":"222'344","reported_seq":4157,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:55.015260+0000","last_change":"2024-09-14T07:11:54.846341+0000","last_active":"2024-09-14T07:11:55.015260+0000","last_peered":"2024-09-14T07:11:55.015260+0000","last_clean":"2024-09-14T07:11:55.015260+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:11:55.015260+0000","last_undegraded":"2024-09-14T07:11:55.015260+0000","last_fullsized":"2024-09-14T07:11:55.015260+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:50.694221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023414690000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.e","version":"220'446","reported_seq":17462,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.867671+0000","last_change":"2024-09-14T07:11:54.868113+0000","last_active":"2024-09-14T07:11:54.867671+0000","last_peered":"2024-09-14T07:11:54.867671+0000","last_clean":"2024-09-14T07:11:54.867671+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:11:54.867671+0000","last_undegraded":"2024-09-14T07:11:54.867671+0000","last_fullsized":"2024-09-14T07:11:54.867671+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:54:55.683061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001943455,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.f","version":"221'485","reported_seq":8549,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866400+0000","last_change":"2024-09-14T07:11:54.866888+0000","last_active":"2024-09-14T07:11:54.866400+0000","last_peered":"2024-09-14T07:11:54.866400+0000","last_clean":"2024-09-14T07:11:54.866400+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:11:54.866400+0000","last_undegraded":"2024-09-14T07:11:54.866400+0000","last_fullsized":"2024-09-14T07:11:54.866400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:57:48.062313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086905399999999996,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.10","version":"222'463","reported_seq":8163,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.866954+0000","last_change":"2024-09-14T07:11:54.866954+0000","last_active":"2024-09-14T07:11:54.866954+0000","last_peered":"2024-09-14T07:11:54.866954+0000","last_clean":"2024-09-14T07:11:54.866954+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:11:54.866954+0000","last_undegraded":"2024-09-14T07:11:54.866954+0000","last_fullsized":"2024-09-14T07:11:54.866954+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:10:12.283798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010057550000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.11","version":"222'560","reported_seq":9546,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.846947+0000","last_change":"2024-09-14T07:11:54.846947+0000","last_active":"2024-09-14T07:11:54.846947+0000","last_peered":"2024-09-14T07:11:54.846947+0000","last_clean":"2024-09-14T07:11:54.846947+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:11:54.846947+0000","last_undegraded":"2024-09-14T07:11:54.846947+0000","last_fullsized":"2024-09-14T07:11:54.846947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:29:32.956389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002649775,"stat_sum":{"num_bytes":3663872,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":437,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.12","version":"222'559","reported_seq":7319,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.844940+0000","last_change":"2024-09-14T07:11:54.844940+0000","last_active":"2024-09-14T07:11:54.844940+0000","last_peered":"2024-09-14T07:11:54.844940+0000","last_clean":"2024-09-14T07:11:54.844940+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:11:54.844940+0000","last_undegraded":"2024-09-14T07:11:54.844940+0000","last_fullsized":"2024-09-14T07:11:54.844940+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:38:40.357896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001463347,"stat_sum":{"num_bytes":10882048,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":429,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]},{"pgid":"3.13","version":"222'383","reported_seq":4978,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-14T07:11:54.844796+0000","last_change":"2024-09-14T07:11:54.844796+0000","last_active":"2024-09-14T07:11:54.844796+0000","last_peered":"2024-09-14T07:11:54.844796+0000","last_clean":"2024-09-14T07:11:54.844796+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:11:54.844796+0000","last_undegraded":"2024-09-14T07:11:54.844796+0000","last_fullsized":"2024-09-14T07:11:54.844796+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:55:43.278292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001280376,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":35232275,"num_objects":266,"num_object_clones":0,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":249,"num_read":341675,"num_read_kb":1116521,"num_write":22066,"num_write_kb":329920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":1175655,"data_compressed":233311,"data_compressed_allocated":323584,"data_compressed_original":1130496,"omap_allocated":12989,"internal_metadata":0},"log_size":19477,"ondisk_log_size":19477,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82424,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289416,"statfs":{"total":96636764160,"available":96552361984,"internally_reserved":0,"allocated":2691072,"data_stored":1117457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607609,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51924,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319916,"statfs":{"total":96636764160,"available":96583593984,"internally_reserved":0,"allocated":2744320,"data_stored":1300489,"data_compressed":32306,"data_compressed_allocated":45056,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607609,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70360,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301480,"statfs":{"total":96636764160,"available":96564715520,"internally_reserved":0,"allocated":3141632,"data_stored":2593879,"data_compressed":273623,"data_compressed_allocated":376832,"data_compressed_original":1433600,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607610,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47956,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323884,"statfs":{"total":96636764160,"available":96587657216,"internally_reserved":0,"allocated":2940928,"data_stored":1816863,"data_compressed":159177,"data_compressed_allocated":212992,"data_compressed_original":684032,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607610,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47852,"kb_used_data":2764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323988,"statfs":{"total":96636764160,"available":96587763712,"internally_reserved":0,"allocated":2830336,"data_stored":1431351,"data_compressed":112714,"data_compressed_allocated":135168,"data_compressed_original":315392,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164868,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35904,"kb_used_data":2668,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335936,"statfs":{"total":96636764160,"available":96599998464,"internally_reserved":0,"allocated":2732032,"data_stored":1707825,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":209920,"data_compressed":46844,"data_compressed_allocated":61440,"data_compressed_original":200704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":406577,"data_compressed":68930,"data_compressed_allocated":98304,"data_compressed_original":389120,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":370704,"data_compressed":85231,"data_compressed_allocated":118784,"data_compressed_original":360448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":188416,"data_compressed":32306,"data_compressed_allocated":45056,"data_compressed_original":180224,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:11:59.902 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T07:11:59.903 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:11:59.903 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-14T07:11:59.903 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:11:59.903 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:11:59.903 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-14T07:12:00.038 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-14T07:12:00.038 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "" 2024-09-14T07:12:00.038 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-14T07:12:05.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T07:12:05.201 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a: {} 2024-09-14T07:12:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:12:05.214 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b: {} 2024-09-14T07:12:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:12:05.224 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c: {} 2024-09-14T07:12:05.224 INFO:teuthology.orchestra.run.smithi005.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:12:05.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T07:12:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T07:12:06.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T07:12:07.257 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T07:12:07.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T07:12:08.278 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T07:12:08.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T07:12:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T07:12:09.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T07:12:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T07:12:10.320 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T07:12:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T07:12:11.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T07:12:12.310 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T07:12:12.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T07:12:13.314 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T07:12:13.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T07:12:14.331 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T07:12:14.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T07:12:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T07:12:15.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T07:12:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T07:12:16.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T07:12:17.353 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T07:12:17.381 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T07:12:17.381 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T07:12:17.381 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T07:12:17.382 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T07:12:17.382 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T07:12:17.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:12:17.700 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:17.700 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":238,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:12:17.351541+0000","last_up_change":"2024-09-14T07:11:42.075125+0000","last_in_change":"2024-09-14T07:10:52.180916+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T07:07:31.246122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T07:07:34.989638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T07:07:45.828508+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":223,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":182,"up_from":209,"up_thru":216,"down_at":184,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6817","nonce":2587782519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2587782519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2587782519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2587782519}]},"public_addr":"172.21.15.5:6817/2587782519","cluster_addr":"172.21.15.5:6819/2587782519","heartbeat_back_addr":"172.21.15.5:6823/2587782519","heartbeat_front_addr":"172.21.15.5:6821/2587782519","state":["exists","up"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6801","nonce":1704459401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6803","nonce":1704459401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6807","nonce":1704459401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6805","nonce":1704459401}]},"public_addr":"172.21.15.5:6801/1704459401","cluster_addr":"172.21.15.5:6803/1704459401","heartbeat_back_addr":"172.21.15.5:6807/1704459401","heartbeat_front_addr":"172.21.15.5:6805/1704459401","state":["exists","up"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":230,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6809","nonce":2565128645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6811","nonce":2565128645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6815","nonce":2565128645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6813","nonce":2565128645}]},"public_addr":"172.21.15.5:6809/2565128645","cluster_addr":"172.21.15.5:6811/2565128645","heartbeat_back_addr":"172.21.15.5:6815/2565128645","heartbeat_front_addr":"172.21.15.5:6813/2565128645","state":["exists","up"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6801","nonce":2226792373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6803","nonce":2226792373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6807","nonce":2226792373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6805","nonce":2226792373}]},"public_addr":"172.21.15.110:6801/2226792373","cluster_addr":"172.21.15.110:6803/2226792373","heartbeat_back_addr":"172.21.15.110:6807/2226792373","heartbeat_front_addr":"172.21.15.110:6805/2226792373","state":["exists","up"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6817","nonce":940255113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6819","nonce":940255113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6823","nonce":940255113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6821","nonce":940255113}]},"public_addr":"172.21.15.110:6817/940255113","cluster_addr":"172.21.15.110:6819/940255113","heartbeat_back_addr":"172.21.15.110:6823/940255113","heartbeat_front_addr":"172.21.15.110:6821/940255113","state":["exists","up"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6809","nonce":4012173203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6811","nonce":4012173203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6815","nonce":4012173203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6813","nonce":4012173203}]},"public_addr":"172.21.15.110:6809/4012173203","cluster_addr":"172.21.15.110:6811/4012173203","heartbeat_back_addr":"172.21.15.110:6815/4012173203","heartbeat_front_addr":"172.21.15.110:6813/4012173203","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.952391+0000","dead_epoch":184},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.101881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.869317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.961339+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.296733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.723820+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1a","osds":[4,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:12:17.719 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T07:12:17.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:12:18.028 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:18.028 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":238,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:12:17.351541+0000","last_up_change":"2024-09-14T07:11:42.075125+0000","last_in_change":"2024-09-14T07:10:52.180916+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T07:07:31.246122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T07:07:34.989638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T07:07:45.828508+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":223,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":182,"up_from":209,"up_thru":216,"down_at":184,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6817","nonce":2587782519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2587782519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2587782519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2587782519}]},"public_addr":"172.21.15.5:6817/2587782519","cluster_addr":"172.21.15.5:6819/2587782519","heartbeat_back_addr":"172.21.15.5:6823/2587782519","heartbeat_front_addr":"172.21.15.5:6821/2587782519","state":["exists","up"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6801","nonce":1704459401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6803","nonce":1704459401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6807","nonce":1704459401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6805","nonce":1704459401}]},"public_addr":"172.21.15.5:6801/1704459401","cluster_addr":"172.21.15.5:6803/1704459401","heartbeat_back_addr":"172.21.15.5:6807/1704459401","heartbeat_front_addr":"172.21.15.5:6805/1704459401","state":["exists","up"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":230,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6809","nonce":2565128645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6811","nonce":2565128645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6815","nonce":2565128645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6813","nonce":2565128645}]},"public_addr":"172.21.15.5:6809/2565128645","cluster_addr":"172.21.15.5:6811/2565128645","heartbeat_back_addr":"172.21.15.5:6815/2565128645","heartbeat_front_addr":"172.21.15.5:6813/2565128645","state":["exists","up"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6801","nonce":2226792373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6803","nonce":2226792373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6807","nonce":2226792373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6805","nonce":2226792373}]},"public_addr":"172.21.15.110:6801/2226792373","cluster_addr":"172.21.15.110:6803/2226792373","heartbeat_back_addr":"172.21.15.110:6807/2226792373","heartbeat_front_addr":"172.21.15.110:6805/2226792373","state":["exists","up"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6817","nonce":940255113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6819","nonce":940255113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6823","nonce":940255113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6821","nonce":940255113}]},"public_addr":"172.21.15.110:6817/940255113","cluster_addr":"172.21.15.110:6819/940255113","heartbeat_back_addr":"172.21.15.110:6823/940255113","heartbeat_front_addr":"172.21.15.110:6821/940255113","state":["exists","up"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6809","nonce":4012173203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6811","nonce":4012173203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6815","nonce":4012173203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6813","nonce":4012173203}]},"public_addr":"172.21.15.110:6809/4012173203","cluster_addr":"172.21.15.110:6811/4012173203","heartbeat_back_addr":"172.21.15.110:6815/4012173203","heartbeat_front_addr":"172.21.15.110:6813/4012173203","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.952391+0000","dead_epoch":184},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.101881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.869317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.961339+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.296733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.723820+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1a","osds":[4,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:12:18.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T07:12:18.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T07:12:18.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T07:12:18.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T07:12:18.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T07:12:18.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T07:12:18.222 INFO:teuthology.orchestra.run.smithi005.stdout:897648164873 2024-09-14T07:12:18.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T07:12:18.248 INFO:teuthology.orchestra.run.smithi005.stdout:51539607615 2024-09-14T07:12:18.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T07:12:18.277 INFO:teuthology.orchestra.run.smithi005.stdout:55834574913 2024-09-14T07:12:18.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T07:12:18.288 INFO:teuthology.orchestra.run.smithi005.stdout:51539607615 2024-09-14T07:12:18.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T07:12:18.305 INFO:teuthology.orchestra.run.smithi005.stdout:51539607614 2024-09-14T07:12:18.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T07:12:18.308 INFO:teuthology.orchestra.run.smithi005.stdout:51539607614 2024-09-14T07:12:18.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T07:12:18.633 INFO:teuthology.orchestra.run.smithi005.stdout:51539607614 2024-09-14T07:12:18.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607615 got 51539607614 for osd.1 2024-09-14T07:12:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:51539607613 2024-09-14T07:12:18.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607614 got 51539607613 for osd.3 2024-09-14T07:12:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:897648164872 2024-09-14T07:12:18.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 897648164873 got 897648164872 for osd.0 2024-09-14T07:12:18.797 INFO:teuthology.orchestra.run.smithi005.stdout:51539607614 2024-09-14T07:12:18.797 INFO:teuthology.orchestra.run.smithi005.stdout:55834574911 2024-09-14T07:12:18.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607615 got 51539607614 for osd.2 2024-09-14T07:12:18.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574913 got 55834574911 for osd.4 2024-09-14T07:12:18.826 INFO:teuthology.orchestra.run.smithi005.stdout:51539607613 2024-09-14T07:12:18.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607614 got 51539607613 for osd.5 2024-09-14T07:12:19.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T07:12:19.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T07:12:19.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T07:12:19.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T07:12:19.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T07:12:19.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T07:12:20.009 INFO:teuthology.orchestra.run.smithi005.stdout:51539607615 2024-09-14T07:12:20.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607615 got 51539607615 for osd.1 2024-09-14T07:12:20.029 DEBUG:teuthology.parallel:result is None 2024-09-14T07:12:20.105 INFO:teuthology.orchestra.run.smithi005.stdout:51539607614 2024-09-14T07:12:20.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607614 got 51539607614 for osd.3 2024-09-14T07:12:20.123 DEBUG:teuthology.parallel:result is None 2024-09-14T07:12:20.154 INFO:teuthology.orchestra.run.smithi005.stdout:51539607615 2024-09-14T07:12:20.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607615 got 51539607615 for osd.2 2024-09-14T07:12:20.172 DEBUG:teuthology.parallel:result is None 2024-09-14T07:12:20.210 INFO:teuthology.orchestra.run.smithi005.stdout:897648164873 2024-09-14T07:12:20.216 INFO:teuthology.orchestra.run.smithi005.stdout:55834574913 2024-09-14T07:12:20.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 897648164873 got 897648164873 for osd.0 2024-09-14T07:12:20.230 DEBUG:teuthology.parallel:result is None 2024-09-14T07:12:20.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574913 got 55834574913 for osd.4 2024-09-14T07:12:20.235 DEBUG:teuthology.parallel:result is None 2024-09-14T07:12:20.288 INFO:teuthology.orchestra.run.smithi005.stdout:51539607614 2024-09-14T07:12:20.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607614 got 51539607614 for osd.5 2024-09-14T07:12:20.305 DEBUG:teuthology.parallel:result is None 2024-09-14T07:12:20.305 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T07:12:20.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:20.613 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-14T07:12:19.279044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336628,"kb_used_data":16924,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894412,"statfs":{"total":579820584960,"available":579475877888,"internally_reserved":0,"allocated":17330176,"data_stored":8328346,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7017,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537789+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:17.537789+0000","last_peered":"2024-09-14T07:12:17.537789+0000","last_clean":"2024-09-14T07:12:17.537789+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:17.537789+0000","last_undegraded":"2024-09-14T07:12:17.537789+0000","last_fullsized":"2024-09-14T07:12:17.537789+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13052,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358582+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:17.358582+0000","last_peered":"2024-09-14T07:12:17.358582+0000","last_clean":"2024-09-14T07:12:17.358582+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:17.358582+0000","last_undegraded":"2024-09-14T07:12:17.358582+0000","last_fullsized":"2024-09-14T07:12:17.358582+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16744,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534139+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:17.534139+0000","last_peered":"2024-09-14T07:12:17.534139+0000","last_clean":"2024-09-14T07:12:17.534139+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:17.534139+0000","last_undegraded":"2024-09-14T07:12:17.534139+0000","last_fullsized":"2024-09-14T07:12:17.534139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20879,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533818+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:17.533818+0000","last_peered":"2024-09-14T07:12:17.533818+0000","last_clean":"2024-09-14T07:12:17.533818+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:17.533818+0000","last_undegraded":"2024-09-14T07:12:17.533818+0000","last_fullsized":"2024-09-14T07:12:17.533818+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8735,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537135+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:17.537135+0000","last_peered":"2024-09-14T07:12:17.537135+0000","last_clean":"2024-09-14T07:12:17.537135+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:17.537135+0000","last_undegraded":"2024-09-14T07:12:17.537135+0000","last_fullsized":"2024-09-14T07:12:17.537135+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30505,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358336+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:17.358336+0000","last_peered":"2024-09-14T07:12:17.358336+0000","last_clean":"2024-09-14T07:12:17.358336+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:17.358336+0000","last_undegraded":"2024-09-14T07:12:17.358336+0000","last_fullsized":"2024-09-14T07:12:17.358336+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10671,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.032828+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.032828+0000","last_peered":"2024-09-14T07:12:18.032828+0000","last_clean":"2024-09-14T07:12:18.032828+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.032828+0000","last_undegraded":"2024-09-14T07:12:18.032828+0000","last_fullsized":"2024-09-14T07:12:18.032828+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5485,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358550+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:17.358550+0000","last_peered":"2024-09-14T07:12:17.358550+0000","last_clean":"2024-09-14T07:12:17.358550+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:17.358550+0000","last_undegraded":"2024-09-14T07:12:17.358550+0000","last_fullsized":"2024-09-14T07:12:17.358550+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3835,"reported_epoch":238,"state":"peering","last_fresh":"2024-09-14T07:12:17.537422+0000","last_change":"2024-09-14T07:12:17.537422+0000","last_active":"2024-09-14T07:12:17.537422+0000","last_peered":"2024-09-14T07:11:50.005660+0000","last_clean":"2024-09-14T07:11:50.005660+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:12:17.537422+0000","last_undegraded":"2024-09-14T07:12:17.537422+0000","last_fullsized":"2024-09-14T07:12:17.537422+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10082,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533947+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:17.533947+0000","last_peered":"2024-09-14T07:12:17.533947+0000","last_clean":"2024-09-14T07:12:17.533947+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:17.533947+0000","last_undegraded":"2024-09-14T07:12:17.533947+0000","last_fullsized":"2024-09-14T07:12:17.533947+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7142,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540455+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:17.540455+0000","last_peered":"2024-09-14T07:12:17.540455+0000","last_clean":"2024-09-14T07:12:17.540455+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:17.540455+0000","last_undegraded":"2024-09-14T07:12:17.540455+0000","last_fullsized":"2024-09-14T07:12:17.540455+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11147,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540295+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:17.540295+0000","last_peered":"2024-09-14T07:12:17.540295+0000","last_clean":"2024-09-14T07:12:17.540295+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:17.540295+0000","last_undegraded":"2024-09-14T07:12:17.540295+0000","last_fullsized":"2024-09-14T07:12:17.540295+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":471,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540882+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:17.540882+0000","last_peered":"2024-09-14T07:12:17.540882+0000","last_clean":"2024-09-14T07:12:17.540882+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:17.540882+0000","last_undegraded":"2024-09-14T07:12:17.540882+0000","last_fullsized":"2024-09-14T07:12:17.540882+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12873,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537279+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:17.537279+0000","last_peered":"2024-09-14T07:12:17.537279+0000","last_clean":"2024-09-14T07:12:17.537279+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:17.537279+0000","last_undegraded":"2024-09-14T07:12:17.537279+0000","last_fullsized":"2024-09-14T07:12:17.537279+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033299+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.033299+0000","last_peered":"2024-09-14T07:12:18.033299+0000","last_clean":"2024-09-14T07:12:18.033299+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.033299+0000","last_undegraded":"2024-09-14T07:12:18.033299+0000","last_fullsized":"2024-09-14T07:12:18.033299+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8375,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358559+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:17.358559+0000","last_peered":"2024-09-14T07:12:17.358559+0000","last_clean":"2024-09-14T07:12:17.358559+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:17.358559+0000","last_undegraded":"2024-09-14T07:12:17.358559+0000","last_fullsized":"2024-09-14T07:12:17.358559+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":480,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540776+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:17.540776+0000","last_peered":"2024-09-14T07:12:17.540776+0000","last_clean":"2024-09-14T07:12:17.540776+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:17.540776+0000","last_undegraded":"2024-09-14T07:12:17.540776+0000","last_fullsized":"2024-09-14T07:12:17.540776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6737,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033013+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.033013+0000","last_peered":"2024-09-14T07:12:18.033013+0000","last_clean":"2024-09-14T07:12:18.033013+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.033013+0000","last_undegraded":"2024-09-14T07:12:18.033013+0000","last_fullsized":"2024-09-14T07:12:18.033013+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":174,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033209+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.033209+0000","last_peered":"2024-09-14T07:12:18.033209+0000","last_clean":"2024-09-14T07:12:18.033209+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.033209+0000","last_undegraded":"2024-09-14T07:12:18.033209+0000","last_fullsized":"2024-09-14T07:12:18.033209+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4099,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534205+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:17.534205+0000","last_peered":"2024-09-14T07:12:17.534205+0000","last_clean":"2024-09-14T07:12:17.534205+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:17.534205+0000","last_undegraded":"2024-09-14T07:12:17.534205+0000","last_fullsized":"2024-09-14T07:12:17.534205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10158,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358545+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:17.358545+0000","last_peered":"2024-09-14T07:12:17.358545+0000","last_clean":"2024-09-14T07:12:17.358545+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:17.358545+0000","last_undegraded":"2024-09-14T07:12:17.358545+0000","last_fullsized":"2024-09-14T07:12:17.358545+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":481,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533748+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:17.533748+0000","last_peered":"2024-09-14T07:12:17.533748+0000","last_clean":"2024-09-14T07:12:17.533748+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:17.533748+0000","last_undegraded":"2024-09-14T07:12:17.533748+0000","last_fullsized":"2024-09-14T07:12:17.533748+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6417,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033071+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.033071+0000","last_peered":"2024-09-14T07:12:18.033071+0000","last_clean":"2024-09-14T07:12:18.033071+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.033071+0000","last_undegraded":"2024-09-14T07:12:18.033071+0000","last_fullsized":"2024-09-14T07:12:18.033071+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":288,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358550+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:17.358550+0000","last_peered":"2024-09-14T07:12:17.358550+0000","last_clean":"2024-09-14T07:12:17.358550+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:17.358550+0000","last_undegraded":"2024-09-14T07:12:17.358550+0000","last_fullsized":"2024-09-14T07:12:17.358550+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13048,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537050+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:17.537050+0000","last_peered":"2024-09-14T07:12:17.537050+0000","last_clean":"2024-09-14T07:12:17.537050+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:17.537050+0000","last_undegraded":"2024-09-14T07:12:17.537050+0000","last_fullsized":"2024-09-14T07:12:17.537050+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":471,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540144+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:17.540144+0000","last_peered":"2024-09-14T07:12:17.540144+0000","last_clean":"2024-09-14T07:12:17.540144+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:17.540144+0000","last_undegraded":"2024-09-14T07:12:17.540144+0000","last_fullsized":"2024-09-14T07:12:17.540144+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7916,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033013+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.033013+0000","last_peered":"2024-09-14T07:12:18.033013+0000","last_clean":"2024-09-14T07:12:18.033013+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.033013+0000","last_undegraded":"2024-09-14T07:12:18.033013+0000","last_fullsized":"2024-09-14T07:12:18.033013+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":480,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533827+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:17.533827+0000","last_peered":"2024-09-14T07:12:17.533827+0000","last_clean":"2024-09-14T07:12:17.533827+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:17.533827+0000","last_undegraded":"2024-09-14T07:12:17.533827+0000","last_fullsized":"2024-09-14T07:12:17.533827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":583,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540182+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:17.540182+0000","last_peered":"2024-09-14T07:12:17.540182+0000","last_clean":"2024-09-14T07:12:17.540182+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:17.540182+0000","last_undegraded":"2024-09-14T07:12:17.540182+0000","last_fullsized":"2024-09-14T07:12:17.540182+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10306,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358734+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:17.358734+0000","last_peered":"2024-09-14T07:12:17.358734+0000","last_clean":"2024-09-14T07:12:17.358734+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:17.358734+0000","last_undegraded":"2024-09-14T07:12:17.358734+0000","last_fullsized":"2024-09-14T07:12:17.358734+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8432,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358479+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:17.358479+0000","last_peered":"2024-09-14T07:12:17.358479+0000","last_clean":"2024-09-14T07:12:17.358479+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:17.358479+0000","last_undegraded":"2024-09-14T07:12:17.358479+0000","last_fullsized":"2024-09-14T07:12:17.358479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8266,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.541783+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:17.541783+0000","last_peered":"2024-09-14T07:12:17.541783+0000","last_clean":"2024-09-14T07:12:17.541783+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:17.541783+0000","last_undegraded":"2024-09-14T07:12:17.541783+0000","last_fullsized":"2024-09-14T07:12:17.541783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8179,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033032+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.033032+0000","last_peered":"2024-09-14T07:12:18.033032+0000","last_clean":"2024-09-14T07:12:18.033032+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.033032+0000","last_undegraded":"2024-09-14T07:12:18.033032+0000","last_fullsized":"2024-09-14T07:12:18.033032+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15526,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534042+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:17.534042+0000","last_peered":"2024-09-14T07:12:17.534042+0000","last_clean":"2024-09-14T07:12:17.534042+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:17.534042+0000","last_undegraded":"2024-09-14T07:12:17.534042+0000","last_fullsized":"2024-09-14T07:12:17.534042+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4193,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540774+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:17.540774+0000","last_peered":"2024-09-14T07:12:17.540774+0000","last_clean":"2024-09-14T07:12:17.540774+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:17.540774+0000","last_undegraded":"2024-09-14T07:12:17.540774+0000","last_fullsized":"2024-09-14T07:12:17.540774+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17498,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534000+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:17.534000+0000","last_peered":"2024-09-14T07:12:17.534000+0000","last_clean":"2024-09-14T07:12:17.534000+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:17.534000+0000","last_undegraded":"2024-09-14T07:12:17.534000+0000","last_fullsized":"2024-09-14T07:12:17.534000+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8585,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534336+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:17.534336+0000","last_peered":"2024-09-14T07:12:17.534336+0000","last_clean":"2024-09-14T07:12:17.534336+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:17.534336+0000","last_undegraded":"2024-09-14T07:12:17.534336+0000","last_fullsized":"2024-09-14T07:12:17.534336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8199,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533637+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:17.533637+0000","last_peered":"2024-09-14T07:12:17.533637+0000","last_clean":"2024-09-14T07:12:17.533637+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:17.533637+0000","last_undegraded":"2024-09-14T07:12:17.533637+0000","last_fullsized":"2024-09-14T07:12:17.533637+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9583,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540245+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:17.540245+0000","last_peered":"2024-09-14T07:12:17.540245+0000","last_clean":"2024-09-14T07:12:17.540245+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:17.540245+0000","last_undegraded":"2024-09-14T07:12:17.540245+0000","last_fullsized":"2024-09-14T07:12:17.540245+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7328,"reported_epoch":238,"state":"peering","last_fresh":"2024-09-14T07:12:17.537546+0000","last_change":"2024-09-14T07:12:17.537546+0000","last_active":"2024-09-14T07:12:17.537546+0000","last_peered":"2024-09-14T07:11:55.928070+0000","last_clean":"2024-09-14T07:11:55.928070+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:12:17.537546+0000","last_undegraded":"2024-09-14T07:12:17.537546+0000","last_fullsized":"2024-09-14T07:12:17.537546+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5014,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540303+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:17.540303+0000","last_peered":"2024-09-14T07:12:17.540303+0000","last_clean":"2024-09-14T07:12:17.540303+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:17.540303+0000","last_undegraded":"2024-09-14T07:12:17.540303+0000","last_fullsized":"2024-09-14T07:12:17.540303+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82576,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289264,"statfs":{"total":96636764160,"available":96552206336,"internally_reserved":0,"allocated":2875392,"data_stored":1192103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607614,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52052,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319788,"statfs":{"total":96636764160,"available":96583462912,"internally_reserved":0,"allocated":2875392,"data_stored":1192103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607614,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70152,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301688,"statfs":{"total":96636764160,"available":96564928512,"internally_reserved":0,"allocated":2928640,"data_stored":1782452,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607615,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47884,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323956,"statfs":{"total":96636764160,"available":96587730944,"internally_reserved":0,"allocated":2871296,"data_stored":1192084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607615,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47872,"kb_used_data":2796,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323968,"statfs":{"total":96636764160,"available":96587743232,"internally_reserved":0,"allocated":2863104,"data_stored":1187133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":2848,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":2916352,"data_stored":1782471,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:20.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:20.906 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:20.922 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-14T07:12:19.279044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336628,"kb_used_data":16924,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894412,"statfs":{"total":579820584960,"available":579475877888,"internally_reserved":0,"allocated":17330176,"data_stored":8328346,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001074"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7017,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537789+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:17.537789+0000","last_peered":"2024-09-14T07:12:17.537789+0000","last_clean":"2024-09-14T07:12:17.537789+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:17.537789+0000","last_undegraded":"2024-09-14T07:12:17.537789+0000","last_fullsized":"2024-09-14T07:12:17.537789+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13052,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358582+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:17.358582+0000","last_peered":"2024-09-14T07:12:17.358582+0000","last_clean":"2024-09-14T07:12:17.358582+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:17.358582+0000","last_undegraded":"2024-09-14T07:12:17.358582+0000","last_fullsized":"2024-09-14T07:12:17.358582+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16744,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534139+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:17.534139+0000","last_peered":"2024-09-14T07:12:17.534139+0000","last_clean":"2024-09-14T07:12:17.534139+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:17.534139+0000","last_undegraded":"2024-09-14T07:12:17.534139+0000","last_fullsized":"2024-09-14T07:12:17.534139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20879,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533818+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:17.533818+0000","last_peered":"2024-09-14T07:12:17.533818+0000","last_clean":"2024-09-14T07:12:17.533818+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:17.533818+0000","last_undegraded":"2024-09-14T07:12:17.533818+0000","last_fullsized":"2024-09-14T07:12:17.533818+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8735,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537135+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:17.537135+0000","last_peered":"2024-09-14T07:12:17.537135+0000","last_clean":"2024-09-14T07:12:17.537135+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:17.537135+0000","last_undegraded":"2024-09-14T07:12:17.537135+0000","last_fullsized":"2024-09-14T07:12:17.537135+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30505,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358336+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:17.358336+0000","last_peered":"2024-09-14T07:12:17.358336+0000","last_clean":"2024-09-14T07:12:17.358336+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:17.358336+0000","last_undegraded":"2024-09-14T07:12:17.358336+0000","last_fullsized":"2024-09-14T07:12:17.358336+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10671,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.032828+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.032828+0000","last_peered":"2024-09-14T07:12:18.032828+0000","last_clean":"2024-09-14T07:12:18.032828+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.032828+0000","last_undegraded":"2024-09-14T07:12:18.032828+0000","last_fullsized":"2024-09-14T07:12:18.032828+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5485,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358550+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:17.358550+0000","last_peered":"2024-09-14T07:12:17.358550+0000","last_clean":"2024-09-14T07:12:17.358550+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:17.358550+0000","last_undegraded":"2024-09-14T07:12:17.358550+0000","last_fullsized":"2024-09-14T07:12:17.358550+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3835,"reported_epoch":238,"state":"peering","last_fresh":"2024-09-14T07:12:17.537422+0000","last_change":"2024-09-14T07:12:17.537422+0000","last_active":"2024-09-14T07:12:17.537422+0000","last_peered":"2024-09-14T07:11:50.005660+0000","last_clean":"2024-09-14T07:11:50.005660+0000","last_became_active":"2024-09-14T07:11:42.793873+0000","last_became_peered":"2024-09-14T07:11:42.793873+0000","last_unstale":"2024-09-14T07:12:17.537422+0000","last_undegraded":"2024-09-14T07:12:17.537422+0000","last_fullsized":"2024-09-14T07:12:17.537422+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10082,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533947+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:17.533947+0000","last_peered":"2024-09-14T07:12:17.533947+0000","last_clean":"2024-09-14T07:12:17.533947+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:17.533947+0000","last_undegraded":"2024-09-14T07:12:17.533947+0000","last_fullsized":"2024-09-14T07:12:17.533947+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7142,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540455+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:17.540455+0000","last_peered":"2024-09-14T07:12:17.540455+0000","last_clean":"2024-09-14T07:12:17.540455+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:17.540455+0000","last_undegraded":"2024-09-14T07:12:17.540455+0000","last_fullsized":"2024-09-14T07:12:17.540455+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11147,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540295+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:17.540295+0000","last_peered":"2024-09-14T07:12:17.540295+0000","last_clean":"2024-09-14T07:12:17.540295+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:17.540295+0000","last_undegraded":"2024-09-14T07:12:17.540295+0000","last_fullsized":"2024-09-14T07:12:17.540295+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":471,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540882+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:17.540882+0000","last_peered":"2024-09-14T07:12:17.540882+0000","last_clean":"2024-09-14T07:12:17.540882+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:17.540882+0000","last_undegraded":"2024-09-14T07:12:17.540882+0000","last_fullsized":"2024-09-14T07:12:17.540882+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12873,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537279+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:17.537279+0000","last_peered":"2024-09-14T07:12:17.537279+0000","last_clean":"2024-09-14T07:12:17.537279+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:17.537279+0000","last_undegraded":"2024-09-14T07:12:17.537279+0000","last_fullsized":"2024-09-14T07:12:17.537279+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033299+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.033299+0000","last_peered":"2024-09-14T07:12:18.033299+0000","last_clean":"2024-09-14T07:12:18.033299+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.033299+0000","last_undegraded":"2024-09-14T07:12:18.033299+0000","last_fullsized":"2024-09-14T07:12:18.033299+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8375,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358559+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:17.358559+0000","last_peered":"2024-09-14T07:12:17.358559+0000","last_clean":"2024-09-14T07:12:17.358559+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:17.358559+0000","last_undegraded":"2024-09-14T07:12:17.358559+0000","last_fullsized":"2024-09-14T07:12:17.358559+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":480,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540776+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:17.540776+0000","last_peered":"2024-09-14T07:12:17.540776+0000","last_clean":"2024-09-14T07:12:17.540776+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:17.540776+0000","last_undegraded":"2024-09-14T07:12:17.540776+0000","last_fullsized":"2024-09-14T07:12:17.540776+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6737,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033013+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.033013+0000","last_peered":"2024-09-14T07:12:18.033013+0000","last_clean":"2024-09-14T07:12:18.033013+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.033013+0000","last_undegraded":"2024-09-14T07:12:18.033013+0000","last_fullsized":"2024-09-14T07:12:18.033013+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":174,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033209+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.033209+0000","last_peered":"2024-09-14T07:12:18.033209+0000","last_clean":"2024-09-14T07:12:18.033209+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.033209+0000","last_undegraded":"2024-09-14T07:12:18.033209+0000","last_fullsized":"2024-09-14T07:12:18.033209+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4099,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534205+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:17.534205+0000","last_peered":"2024-09-14T07:12:17.534205+0000","last_clean":"2024-09-14T07:12:17.534205+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:17.534205+0000","last_undegraded":"2024-09-14T07:12:17.534205+0000","last_fullsized":"2024-09-14T07:12:17.534205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10158,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358545+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:17.358545+0000","last_peered":"2024-09-14T07:12:17.358545+0000","last_clean":"2024-09-14T07:12:17.358545+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:17.358545+0000","last_undegraded":"2024-09-14T07:12:17.358545+0000","last_fullsized":"2024-09-14T07:12:17.358545+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":481,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533748+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:17.533748+0000","last_peered":"2024-09-14T07:12:17.533748+0000","last_clean":"2024-09-14T07:12:17.533748+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:17.533748+0000","last_undegraded":"2024-09-14T07:12:17.533748+0000","last_fullsized":"2024-09-14T07:12:17.533748+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6417,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033071+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.033071+0000","last_peered":"2024-09-14T07:12:18.033071+0000","last_clean":"2024-09-14T07:12:18.033071+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.033071+0000","last_undegraded":"2024-09-14T07:12:18.033071+0000","last_fullsized":"2024-09-14T07:12:18.033071+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":288,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358550+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:17.358550+0000","last_peered":"2024-09-14T07:12:17.358550+0000","last_clean":"2024-09-14T07:12:17.358550+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:17.358550+0000","last_undegraded":"2024-09-14T07:12:17.358550+0000","last_fullsized":"2024-09-14T07:12:17.358550+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13048,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.537050+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:17.537050+0000","last_peered":"2024-09-14T07:12:17.537050+0000","last_clean":"2024-09-14T07:12:17.537050+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:17.537050+0000","last_undegraded":"2024-09-14T07:12:17.537050+0000","last_fullsized":"2024-09-14T07:12:17.537050+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":471,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540144+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:17.540144+0000","last_peered":"2024-09-14T07:12:17.540144+0000","last_clean":"2024-09-14T07:12:17.540144+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:17.540144+0000","last_undegraded":"2024-09-14T07:12:17.540144+0000","last_fullsized":"2024-09-14T07:12:17.540144+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7916,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033013+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.033013+0000","last_peered":"2024-09-14T07:12:18.033013+0000","last_clean":"2024-09-14T07:12:18.033013+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.033013+0000","last_undegraded":"2024-09-14T07:12:18.033013+0000","last_fullsized":"2024-09-14T07:12:18.033013+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":480,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533827+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:17.533827+0000","last_peered":"2024-09-14T07:12:17.533827+0000","last_clean":"2024-09-14T07:12:17.533827+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:17.533827+0000","last_undegraded":"2024-09-14T07:12:17.533827+0000","last_fullsized":"2024-09-14T07:12:17.533827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":583,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540182+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:17.540182+0000","last_peered":"2024-09-14T07:12:17.540182+0000","last_clean":"2024-09-14T07:12:17.540182+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:17.540182+0000","last_undegraded":"2024-09-14T07:12:17.540182+0000","last_fullsized":"2024-09-14T07:12:17.540182+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10306,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358734+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:17.358734+0000","last_peered":"2024-09-14T07:12:17.358734+0000","last_clean":"2024-09-14T07:12:17.358734+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:17.358734+0000","last_undegraded":"2024-09-14T07:12:17.358734+0000","last_fullsized":"2024-09-14T07:12:17.358734+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8432,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358479+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:17.358479+0000","last_peered":"2024-09-14T07:12:17.358479+0000","last_clean":"2024-09-14T07:12:17.358479+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:17.358479+0000","last_undegraded":"2024-09-14T07:12:17.358479+0000","last_fullsized":"2024-09-14T07:12:17.358479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8266,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.541783+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:17.541783+0000","last_peered":"2024-09-14T07:12:17.541783+0000","last_clean":"2024-09-14T07:12:17.541783+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:17.541783+0000","last_undegraded":"2024-09-14T07:12:17.541783+0000","last_fullsized":"2024-09-14T07:12:17.541783+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8179,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.033032+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.033032+0000","last_peered":"2024-09-14T07:12:18.033032+0000","last_clean":"2024-09-14T07:12:18.033032+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.033032+0000","last_undegraded":"2024-09-14T07:12:18.033032+0000","last_fullsized":"2024-09-14T07:12:18.033032+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15526,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534042+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:17.534042+0000","last_peered":"2024-09-14T07:12:17.534042+0000","last_clean":"2024-09-14T07:12:17.534042+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:17.534042+0000","last_undegraded":"2024-09-14T07:12:17.534042+0000","last_fullsized":"2024-09-14T07:12:17.534042+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4193,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540774+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:17.540774+0000","last_peered":"2024-09-14T07:12:17.540774+0000","last_clean":"2024-09-14T07:12:17.540774+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:17.540774+0000","last_undegraded":"2024-09-14T07:12:17.540774+0000","last_fullsized":"2024-09-14T07:12:17.540774+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17498,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534000+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:17.534000+0000","last_peered":"2024-09-14T07:12:17.534000+0000","last_clean":"2024-09-14T07:12:17.534000+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:17.534000+0000","last_undegraded":"2024-09-14T07:12:17.534000+0000","last_fullsized":"2024-09-14T07:12:17.534000+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8585,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.534336+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:17.534336+0000","last_peered":"2024-09-14T07:12:17.534336+0000","last_clean":"2024-09-14T07:12:17.534336+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:17.534336+0000","last_undegraded":"2024-09-14T07:12:17.534336+0000","last_fullsized":"2024-09-14T07:12:17.534336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8199,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.533637+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:17.533637+0000","last_peered":"2024-09-14T07:12:17.533637+0000","last_clean":"2024-09-14T07:12:17.533637+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:17.533637+0000","last_undegraded":"2024-09-14T07:12:17.533637+0000","last_fullsized":"2024-09-14T07:12:17.533637+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9583,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540245+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:17.540245+0000","last_peered":"2024-09-14T07:12:17.540245+0000","last_clean":"2024-09-14T07:12:17.540245+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:17.540245+0000","last_undegraded":"2024-09-14T07:12:17.540245+0000","last_fullsized":"2024-09-14T07:12:17.540245+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7328,"reported_epoch":238,"state":"peering","last_fresh":"2024-09-14T07:12:17.537546+0000","last_change":"2024-09-14T07:12:17.537546+0000","last_active":"2024-09-14T07:12:17.537546+0000","last_peered":"2024-09-14T07:11:55.928070+0000","last_clean":"2024-09-14T07:11:55.928070+0000","last_became_active":"2024-09-14T07:11:42.791226+0000","last_became_peered":"2024-09-14T07:11:42.791226+0000","last_unstale":"2024-09-14T07:12:17.537546+0000","last_undegraded":"2024-09-14T07:12:17.537546+0000","last_fullsized":"2024-09-14T07:12:17.537546+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5014,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.540303+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:17.540303+0000","last_peered":"2024-09-14T07:12:17.540303+0000","last_clean":"2024-09-14T07:12:17.540303+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:17.540303+0000","last_undegraded":"2024-09-14T07:12:17.540303+0000","last_fullsized":"2024-09-14T07:12:17.540303+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82576,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289264,"statfs":{"total":96636764160,"available":96552206336,"internally_reserved":0,"allocated":2875392,"data_stored":1192103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607614,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52052,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319788,"statfs":{"total":96636764160,"available":96583462912,"internally_reserved":0,"allocated":2875392,"data_stored":1192103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607614,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70152,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301688,"statfs":{"total":96636764160,"available":96564928512,"internally_reserved":0,"allocated":2928640,"data_stored":1782452,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607615,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47884,"kb_used_data":2804,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323956,"statfs":{"total":96636764160,"available":96587730944,"internally_reserved":0,"allocated":2871296,"data_stored":1192084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607615,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47872,"kb_used_data":2796,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323968,"statfs":{"total":96636764160,"available":96587743232,"internally_reserved":0,"allocated":2863104,"data_stored":1187133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":2848,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":2916352,"data_stored":1782471,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:20.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T07:12:21.272 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:21.272 INFO:teuthology.orchestra.run.smithi005.stdout:{"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":303,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":239,"num_osds":6,"num_up_osds":6,"osd_up_since":1726297902,"num_in_osds":6,"osd_in_since":1726297852,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":39},{"state_name":"peering","count":2}],"num_pgs":41,"num_pools":3,"num_objects":257,"data_bytes":590406,"bytes_used":344707072,"bytes_avail":579475877888,"bytes_total":579820584960,"inactive_pgs_ratio":0.048780485987663269},"fsmap":{"epoch":1,"btime":"2024-09-14T07:07:18:019388+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-14T07:08:33.220112+0000","services":{}},"progress_events":{}} 2024-09-14T07:12:21.290 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-14T07:12:21.290 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:21.600 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:21.618 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-14T07:12:21.279552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336712,"kb_used_data":16996,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894328,"statfs":{"total":579820584960,"available":579475791872,"internally_reserved":0,"allocated":17403904,"data_stored":8358166,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.943582"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7019,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369738+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:18.369738+0000","last_peered":"2024-09-14T07:12:18.369738+0000","last_clean":"2024-09-14T07:12:18.369738+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:18.369738+0000","last_undegraded":"2024-09-14T07:12:18.369738+0000","last_fullsized":"2024-09-14T07:12:18.369738+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13054,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16746,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372658+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:18.372658+0000","last_peered":"2024-09-14T07:12:18.372658+0000","last_clean":"2024-09-14T07:12:18.372658+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:18.372658+0000","last_undegraded":"2024-09-14T07:12:18.372658+0000","last_fullsized":"2024-09-14T07:12:18.372658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20881,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372972+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:18.372972+0000","last_peered":"2024-09-14T07:12:18.372972+0000","last_clean":"2024-09-14T07:12:18.372972+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:18.372972+0000","last_undegraded":"2024-09-14T07:12:18.372972+0000","last_fullsized":"2024-09-14T07:12:18.372972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8737,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369079+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:18.369079+0000","last_peered":"2024-09-14T07:12:18.369079+0000","last_clean":"2024-09-14T07:12:18.369079+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:18.369079+0000","last_undegraded":"2024-09-14T07:12:18.369079+0000","last_fullsized":"2024-09-14T07:12:18.369079+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30505,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358336+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:17.358336+0000","last_peered":"2024-09-14T07:12:17.358336+0000","last_clean":"2024-09-14T07:12:17.358336+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:17.358336+0000","last_undegraded":"2024-09-14T07:12:17.358336+0000","last_fullsized":"2024-09-14T07:12:17.358336+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10673,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639203+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.639203+0000","last_peered":"2024-09-14T07:12:18.639203+0000","last_clean":"2024-09-14T07:12:18.639203+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.639203+0000","last_undegraded":"2024-09-14T07:12:18.639203+0000","last_fullsized":"2024-09-14T07:12:18.639203+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5487,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368465+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:18.368465+0000","last_peered":"2024-09-14T07:12:18.368465+0000","last_clean":"2024-09-14T07:12:18.368465+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:18.368465+0000","last_undegraded":"2024-09-14T07:12:18.368465+0000","last_fullsized":"2024-09-14T07:12:18.368465+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3844,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.376942+0000","last_change":"2024-09-14T07:12:18.376850+0000","last_active":"2024-09-14T07:12:18.376942+0000","last_peered":"2024-09-14T07:12:18.376942+0000","last_clean":"2024-09-14T07:12:18.376942+0000","last_became_active":"2024-09-14T07:12:18.374908+0000","last_became_peered":"2024-09-14T07:12:18.374908+0000","last_unstale":"2024-09-14T07:12:18.376942+0000","last_undegraded":"2024-09-14T07:12:18.376942+0000","last_fullsized":"2024-09-14T07:12:18.376942+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:14.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10084,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372949+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:18.372949+0000","last_peered":"2024-09-14T07:12:18.372949+0000","last_clean":"2024-09-14T07:12:18.372949+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:18.372949+0000","last_undegraded":"2024-09-14T07:12:18.372949+0000","last_fullsized":"2024-09-14T07:12:18.372949+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7144,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368999+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:18.368999+0000","last_peered":"2024-09-14T07:12:18.368999+0000","last_clean":"2024-09-14T07:12:18.368999+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:18.368999+0000","last_undegraded":"2024-09-14T07:12:18.368999+0000","last_fullsized":"2024-09-14T07:12:18.368999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368752+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:18.368752+0000","last_peered":"2024-09-14T07:12:18.368752+0000","last_clean":"2024-09-14T07:12:18.368752+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:18.368752+0000","last_undegraded":"2024-09-14T07:12:18.368752+0000","last_fullsized":"2024-09-14T07:12:18.368752+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369501+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:18.369501+0000","last_peered":"2024-09-14T07:12:18.369501+0000","last_clean":"2024-09-14T07:12:18.369501+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:18.369501+0000","last_undegraded":"2024-09-14T07:12:18.369501+0000","last_fullsized":"2024-09-14T07:12:18.369501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12875,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368991+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:18.368991+0000","last_peered":"2024-09-14T07:12:18.368991+0000","last_clean":"2024-09-14T07:12:18.368991+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:18.368991+0000","last_undegraded":"2024-09-14T07:12:18.368991+0000","last_fullsized":"2024-09-14T07:12:18.368991+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639779+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.639779+0000","last_peered":"2024-09-14T07:12:18.639779+0000","last_clean":"2024-09-14T07:12:18.639779+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.639779+0000","last_undegraded":"2024-09-14T07:12:18.639779+0000","last_fullsized":"2024-09-14T07:12:18.639779+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8377,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368716+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:18.368716+0000","last_peered":"2024-09-14T07:12:18.368716+0000","last_clean":"2024-09-14T07:12:18.368716+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:18.368716+0000","last_undegraded":"2024-09-14T07:12:18.368716+0000","last_fullsized":"2024-09-14T07:12:18.368716+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369005+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:18.369005+0000","last_peered":"2024-09-14T07:12:18.369005+0000","last_clean":"2024-09-14T07:12:18.369005+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:18.369005+0000","last_undegraded":"2024-09-14T07:12:18.369005+0000","last_fullsized":"2024-09-14T07:12:18.369005+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6739,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639507+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.639507+0000","last_peered":"2024-09-14T07:12:18.639507+0000","last_clean":"2024-09-14T07:12:18.639507+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.639507+0000","last_undegraded":"2024-09-14T07:12:18.639507+0000","last_fullsized":"2024-09-14T07:12:18.639507+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639605+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.639605+0000","last_peered":"2024-09-14T07:12:18.639605+0000","last_clean":"2024-09-14T07:12:18.639605+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.639605+0000","last_undegraded":"2024-09-14T07:12:18.639605+0000","last_fullsized":"2024-09-14T07:12:18.639605+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4101,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372761+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:18.372761+0000","last_peered":"2024-09-14T07:12:18.372761+0000","last_clean":"2024-09-14T07:12:18.372761+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:18.372761+0000","last_undegraded":"2024-09-14T07:12:18.372761+0000","last_fullsized":"2024-09-14T07:12:18.372761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10158,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358545+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:17.358545+0000","last_peered":"2024-09-14T07:12:17.358545+0000","last_clean":"2024-09-14T07:12:17.358545+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:17.358545+0000","last_undegraded":"2024-09-14T07:12:17.358545+0000","last_fullsized":"2024-09-14T07:12:17.358545+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":483,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372478+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:18.372478+0000","last_peered":"2024-09-14T07:12:18.372478+0000","last_clean":"2024-09-14T07:12:18.372478+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:18.372478+0000","last_undegraded":"2024-09-14T07:12:18.372478+0000","last_fullsized":"2024-09-14T07:12:18.372478+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.640028+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.640028+0000","last_peered":"2024-09-14T07:12:18.640028+0000","last_clean":"2024-09-14T07:12:18.640028+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.640028+0000","last_undegraded":"2024-09-14T07:12:18.640028+0000","last_fullsized":"2024-09-14T07:12:18.640028+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":290,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368448+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:18.368448+0000","last_peered":"2024-09-14T07:12:18.368448+0000","last_clean":"2024-09-14T07:12:18.368448+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:18.368448+0000","last_undegraded":"2024-09-14T07:12:18.368448+0000","last_fullsized":"2024-09-14T07:12:18.368448+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13050,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368577+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:18.368577+0000","last_peered":"2024-09-14T07:12:18.368577+0000","last_clean":"2024-09-14T07:12:18.368577+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:18.368577+0000","last_undegraded":"2024-09-14T07:12:18.368577+0000","last_fullsized":"2024-09-14T07:12:18.368577+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368438+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:18.368438+0000","last_peered":"2024-09-14T07:12:18.368438+0000","last_clean":"2024-09-14T07:12:18.368438+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:18.368438+0000","last_undegraded":"2024-09-14T07:12:18.368438+0000","last_fullsized":"2024-09-14T07:12:18.368438+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7918,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639383+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.639383+0000","last_peered":"2024-09-14T07:12:18.639383+0000","last_clean":"2024-09-14T07:12:18.639383+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.639383+0000","last_undegraded":"2024-09-14T07:12:18.639383+0000","last_fullsized":"2024-09-14T07:12:18.639383+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372692+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:18.372692+0000","last_peered":"2024-09-14T07:12:18.372692+0000","last_clean":"2024-09-14T07:12:18.372692+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:18.372692+0000","last_undegraded":"2024-09-14T07:12:18.372692+0000","last_fullsized":"2024-09-14T07:12:18.372692+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368746+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:18.368746+0000","last_peered":"2024-09-14T07:12:18.368746+0000","last_clean":"2024-09-14T07:12:18.368746+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:18.368746+0000","last_undegraded":"2024-09-14T07:12:18.368746+0000","last_fullsized":"2024-09-14T07:12:18.368746+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10308,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8432,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T07:12:17.358479+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:17.358479+0000","last_peered":"2024-09-14T07:12:17.358479+0000","last_clean":"2024-09-14T07:12:17.358479+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:17.358479+0000","last_undegraded":"2024-09-14T07:12:17.358479+0000","last_fullsized":"2024-09-14T07:12:17.358479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8268,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368658+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:18.368658+0000","last_peered":"2024-09-14T07:12:18.368658+0000","last_clean":"2024-09-14T07:12:18.368658+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:18.368658+0000","last_undegraded":"2024-09-14T07:12:18.368658+0000","last_fullsized":"2024-09-14T07:12:18.368658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8181,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639165+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.639165+0000","last_peered":"2024-09-14T07:12:18.639165+0000","last_clean":"2024-09-14T07:12:18.639165+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.639165+0000","last_undegraded":"2024-09-14T07:12:18.639165+0000","last_fullsized":"2024-09-14T07:12:18.639165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15528,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372744+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:18.372744+0000","last_peered":"2024-09-14T07:12:18.372744+0000","last_clean":"2024-09-14T07:12:18.372744+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.372744+0000","last_undegraded":"2024-09-14T07:12:18.372744+0000","last_fullsized":"2024-09-14T07:12:18.372744+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368844+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:18.368844+0000","last_peered":"2024-09-14T07:12:18.368844+0000","last_clean":"2024-09-14T07:12:18.368844+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:18.368844+0000","last_undegraded":"2024-09-14T07:12:18.368844+0000","last_fullsized":"2024-09-14T07:12:18.368844+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17500,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373074+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:18.373074+0000","last_peered":"2024-09-14T07:12:18.373074+0000","last_clean":"2024-09-14T07:12:18.373074+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:18.373074+0000","last_undegraded":"2024-09-14T07:12:18.373074+0000","last_fullsized":"2024-09-14T07:12:18.373074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8588,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373393+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:18.373393+0000","last_peered":"2024-09-14T07:12:18.373393+0000","last_clean":"2024-09-14T07:12:18.373393+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:18.373393+0000","last_undegraded":"2024-09-14T07:12:18.373393+0000","last_fullsized":"2024-09-14T07:12:18.373393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8201,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372420+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:18.372420+0000","last_peered":"2024-09-14T07:12:18.372420+0000","last_clean":"2024-09-14T07:12:18.372420+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:18.372420+0000","last_undegraded":"2024-09-14T07:12:18.372420+0000","last_fullsized":"2024-09-14T07:12:18.372420+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368461+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:18.368461+0000","last_peered":"2024-09-14T07:12:18.368461+0000","last_clean":"2024-09-14T07:12:18.368461+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:18.368461+0000","last_undegraded":"2024-09-14T07:12:18.368461+0000","last_fullsized":"2024-09-14T07:12:18.368461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7337,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.374205+0000","last_change":"2024-09-14T07:12:18.374075+0000","last_active":"2024-09-14T07:12:18.374205+0000","last_peered":"2024-09-14T07:12:18.374205+0000","last_clean":"2024-09-14T07:12:18.374205+0000","last_became_active":"2024-09-14T07:12:18.371429+0000","last_became_peered":"2024-09-14T07:12:18.371429+0000","last_unstale":"2024-09-14T07:12:18.374205+0000","last_undegraded":"2024-09-14T07:12:18.374205+0000","last_fullsized":"2024-09-14T07:12:18.374205+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:24.601874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5017,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369266+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:18.369266+0000","last_peered":"2024-09-14T07:12:18.369266+0000","last_clean":"2024-09-14T07:12:18.369266+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:18.369266+0000","last_undegraded":"2024-09-14T07:12:18.369266+0000","last_fullsized":"2024-09-14T07:12:18.369266+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82576,"kb_used_data":2808,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289264,"statfs":{"total":96636764160,"available":96552206336,"internally_reserved":0,"allocated":2875392,"data_stored":1192103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607615,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70156,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301684,"statfs":{"total":96636764160,"available":96564924416,"internally_reserved":0,"allocated":2940928,"data_stored":1787422,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607616,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":2883584,"data_stored":1197054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607616,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":2860,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":2928640,"data_stored":1787441,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:24.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:24.945 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:24.946 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:24.963 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-14T07:12:23.280261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336728,"kb_used_data":17008,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894312,"statfs":{"total":579820584960,"available":579475775488,"internally_reserved":0,"allocated":17416192,"data_stored":8363136,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.934511"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7019,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369738+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:18.369738+0000","last_peered":"2024-09-14T07:12:18.369738+0000","last_clean":"2024-09-14T07:12:18.369738+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:18.369738+0000","last_undegraded":"2024-09-14T07:12:18.369738+0000","last_fullsized":"2024-09-14T07:12:18.369738+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13054,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16746,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372658+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:18.372658+0000","last_peered":"2024-09-14T07:12:18.372658+0000","last_clean":"2024-09-14T07:12:18.372658+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:18.372658+0000","last_undegraded":"2024-09-14T07:12:18.372658+0000","last_fullsized":"2024-09-14T07:12:18.372658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20881,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372972+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:18.372972+0000","last_peered":"2024-09-14T07:12:18.372972+0000","last_clean":"2024-09-14T07:12:18.372972+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:18.372972+0000","last_undegraded":"2024-09-14T07:12:18.372972+0000","last_fullsized":"2024-09-14T07:12:18.372972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8737,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369079+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:18.369079+0000","last_peered":"2024-09-14T07:12:18.369079+0000","last_clean":"2024-09-14T07:12:18.369079+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:18.369079+0000","last_undegraded":"2024-09-14T07:12:18.369079+0000","last_fullsized":"2024-09-14T07:12:18.369079+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30507,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641648+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:18.641648+0000","last_peered":"2024-09-14T07:12:18.641648+0000","last_clean":"2024-09-14T07:12:18.641648+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.641648+0000","last_undegraded":"2024-09-14T07:12:18.641648+0000","last_fullsized":"2024-09-14T07:12:18.641648+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10673,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639203+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.639203+0000","last_peered":"2024-09-14T07:12:18.639203+0000","last_clean":"2024-09-14T07:12:18.639203+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.639203+0000","last_undegraded":"2024-09-14T07:12:18.639203+0000","last_fullsized":"2024-09-14T07:12:18.639203+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5487,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368465+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:18.368465+0000","last_peered":"2024-09-14T07:12:18.368465+0000","last_clean":"2024-09-14T07:12:18.368465+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:18.368465+0000","last_undegraded":"2024-09-14T07:12:18.368465+0000","last_fullsized":"2024-09-14T07:12:18.368465+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3844,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.376942+0000","last_change":"2024-09-14T07:12:18.376850+0000","last_active":"2024-09-14T07:12:18.376942+0000","last_peered":"2024-09-14T07:12:18.376942+0000","last_clean":"2024-09-14T07:12:18.376942+0000","last_became_active":"2024-09-14T07:12:18.374908+0000","last_became_peered":"2024-09-14T07:12:18.374908+0000","last_unstale":"2024-09-14T07:12:18.376942+0000","last_undegraded":"2024-09-14T07:12:18.376942+0000","last_fullsized":"2024-09-14T07:12:18.376942+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:14.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10084,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372949+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:18.372949+0000","last_peered":"2024-09-14T07:12:18.372949+0000","last_clean":"2024-09-14T07:12:18.372949+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:18.372949+0000","last_undegraded":"2024-09-14T07:12:18.372949+0000","last_fullsized":"2024-09-14T07:12:18.372949+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7144,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368999+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:18.368999+0000","last_peered":"2024-09-14T07:12:18.368999+0000","last_clean":"2024-09-14T07:12:18.368999+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:18.368999+0000","last_undegraded":"2024-09-14T07:12:18.368999+0000","last_fullsized":"2024-09-14T07:12:18.368999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368752+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:18.368752+0000","last_peered":"2024-09-14T07:12:18.368752+0000","last_clean":"2024-09-14T07:12:18.368752+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:18.368752+0000","last_undegraded":"2024-09-14T07:12:18.368752+0000","last_fullsized":"2024-09-14T07:12:18.368752+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369501+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:18.369501+0000","last_peered":"2024-09-14T07:12:18.369501+0000","last_clean":"2024-09-14T07:12:18.369501+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:18.369501+0000","last_undegraded":"2024-09-14T07:12:18.369501+0000","last_fullsized":"2024-09-14T07:12:18.369501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12875,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368991+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:18.368991+0000","last_peered":"2024-09-14T07:12:18.368991+0000","last_clean":"2024-09-14T07:12:18.368991+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:18.368991+0000","last_undegraded":"2024-09-14T07:12:18.368991+0000","last_fullsized":"2024-09-14T07:12:18.368991+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639779+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.639779+0000","last_peered":"2024-09-14T07:12:18.639779+0000","last_clean":"2024-09-14T07:12:18.639779+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.639779+0000","last_undegraded":"2024-09-14T07:12:18.639779+0000","last_fullsized":"2024-09-14T07:12:18.639779+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8377,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368716+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:18.368716+0000","last_peered":"2024-09-14T07:12:18.368716+0000","last_clean":"2024-09-14T07:12:18.368716+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:18.368716+0000","last_undegraded":"2024-09-14T07:12:18.368716+0000","last_fullsized":"2024-09-14T07:12:18.368716+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369005+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:18.369005+0000","last_peered":"2024-09-14T07:12:18.369005+0000","last_clean":"2024-09-14T07:12:18.369005+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:18.369005+0000","last_undegraded":"2024-09-14T07:12:18.369005+0000","last_fullsized":"2024-09-14T07:12:18.369005+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6739,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639507+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.639507+0000","last_peered":"2024-09-14T07:12:18.639507+0000","last_clean":"2024-09-14T07:12:18.639507+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.639507+0000","last_undegraded":"2024-09-14T07:12:18.639507+0000","last_fullsized":"2024-09-14T07:12:18.639507+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639605+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.639605+0000","last_peered":"2024-09-14T07:12:18.639605+0000","last_clean":"2024-09-14T07:12:18.639605+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.639605+0000","last_undegraded":"2024-09-14T07:12:18.639605+0000","last_fullsized":"2024-09-14T07:12:18.639605+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4101,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372761+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:18.372761+0000","last_peered":"2024-09-14T07:12:18.372761+0000","last_clean":"2024-09-14T07:12:18.372761+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:18.372761+0000","last_undegraded":"2024-09-14T07:12:18.372761+0000","last_fullsized":"2024-09-14T07:12:18.372761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10160,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641679+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:18.641679+0000","last_peered":"2024-09-14T07:12:18.641679+0000","last_clean":"2024-09-14T07:12:18.641679+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:18.641679+0000","last_undegraded":"2024-09-14T07:12:18.641679+0000","last_fullsized":"2024-09-14T07:12:18.641679+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":483,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372478+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:18.372478+0000","last_peered":"2024-09-14T07:12:18.372478+0000","last_clean":"2024-09-14T07:12:18.372478+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:18.372478+0000","last_undegraded":"2024-09-14T07:12:18.372478+0000","last_fullsized":"2024-09-14T07:12:18.372478+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.640028+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.640028+0000","last_peered":"2024-09-14T07:12:18.640028+0000","last_clean":"2024-09-14T07:12:18.640028+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.640028+0000","last_undegraded":"2024-09-14T07:12:18.640028+0000","last_fullsized":"2024-09-14T07:12:18.640028+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":290,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368448+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:18.368448+0000","last_peered":"2024-09-14T07:12:18.368448+0000","last_clean":"2024-09-14T07:12:18.368448+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:18.368448+0000","last_undegraded":"2024-09-14T07:12:18.368448+0000","last_fullsized":"2024-09-14T07:12:18.368448+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13050,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368577+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:18.368577+0000","last_peered":"2024-09-14T07:12:18.368577+0000","last_clean":"2024-09-14T07:12:18.368577+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:18.368577+0000","last_undegraded":"2024-09-14T07:12:18.368577+0000","last_fullsized":"2024-09-14T07:12:18.368577+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368438+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:18.368438+0000","last_peered":"2024-09-14T07:12:18.368438+0000","last_clean":"2024-09-14T07:12:18.368438+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:18.368438+0000","last_undegraded":"2024-09-14T07:12:18.368438+0000","last_fullsized":"2024-09-14T07:12:18.368438+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7918,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639383+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.639383+0000","last_peered":"2024-09-14T07:12:18.639383+0000","last_clean":"2024-09-14T07:12:18.639383+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.639383+0000","last_undegraded":"2024-09-14T07:12:18.639383+0000","last_fullsized":"2024-09-14T07:12:18.639383+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372692+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:18.372692+0000","last_peered":"2024-09-14T07:12:18.372692+0000","last_clean":"2024-09-14T07:12:18.372692+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:18.372692+0000","last_undegraded":"2024-09-14T07:12:18.372692+0000","last_fullsized":"2024-09-14T07:12:18.372692+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368746+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:18.368746+0000","last_peered":"2024-09-14T07:12:18.368746+0000","last_clean":"2024-09-14T07:12:18.368746+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:18.368746+0000","last_undegraded":"2024-09-14T07:12:18.368746+0000","last_fullsized":"2024-09-14T07:12:18.368746+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10308,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8434,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641712+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:18.641712+0000","last_peered":"2024-09-14T07:12:18.641712+0000","last_clean":"2024-09-14T07:12:18.641712+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:18.641712+0000","last_undegraded":"2024-09-14T07:12:18.641712+0000","last_fullsized":"2024-09-14T07:12:18.641712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8268,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368658+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:18.368658+0000","last_peered":"2024-09-14T07:12:18.368658+0000","last_clean":"2024-09-14T07:12:18.368658+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:18.368658+0000","last_undegraded":"2024-09-14T07:12:18.368658+0000","last_fullsized":"2024-09-14T07:12:18.368658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8181,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639165+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.639165+0000","last_peered":"2024-09-14T07:12:18.639165+0000","last_clean":"2024-09-14T07:12:18.639165+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.639165+0000","last_undegraded":"2024-09-14T07:12:18.639165+0000","last_fullsized":"2024-09-14T07:12:18.639165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15528,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372744+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:18.372744+0000","last_peered":"2024-09-14T07:12:18.372744+0000","last_clean":"2024-09-14T07:12:18.372744+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.372744+0000","last_undegraded":"2024-09-14T07:12:18.372744+0000","last_fullsized":"2024-09-14T07:12:18.372744+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368844+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:18.368844+0000","last_peered":"2024-09-14T07:12:18.368844+0000","last_clean":"2024-09-14T07:12:18.368844+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:18.368844+0000","last_undegraded":"2024-09-14T07:12:18.368844+0000","last_fullsized":"2024-09-14T07:12:18.368844+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17500,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373074+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:18.373074+0000","last_peered":"2024-09-14T07:12:18.373074+0000","last_clean":"2024-09-14T07:12:18.373074+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:18.373074+0000","last_undegraded":"2024-09-14T07:12:18.373074+0000","last_fullsized":"2024-09-14T07:12:18.373074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8588,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373393+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:18.373393+0000","last_peered":"2024-09-14T07:12:18.373393+0000","last_clean":"2024-09-14T07:12:18.373393+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:18.373393+0000","last_undegraded":"2024-09-14T07:12:18.373393+0000","last_fullsized":"2024-09-14T07:12:18.373393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8201,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372420+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:18.372420+0000","last_peered":"2024-09-14T07:12:18.372420+0000","last_clean":"2024-09-14T07:12:18.372420+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:18.372420+0000","last_undegraded":"2024-09-14T07:12:18.372420+0000","last_fullsized":"2024-09-14T07:12:18.372420+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368461+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:18.368461+0000","last_peered":"2024-09-14T07:12:18.368461+0000","last_clean":"2024-09-14T07:12:18.368461+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:18.368461+0000","last_undegraded":"2024-09-14T07:12:18.368461+0000","last_fullsized":"2024-09-14T07:12:18.368461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7337,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.374205+0000","last_change":"2024-09-14T07:12:18.374075+0000","last_active":"2024-09-14T07:12:18.374205+0000","last_peered":"2024-09-14T07:12:18.374205+0000","last_clean":"2024-09-14T07:12:18.374205+0000","last_became_active":"2024-09-14T07:12:18.371429+0000","last_became_peered":"2024-09-14T07:12:18.371429+0000","last_unstale":"2024-09-14T07:12:18.374205+0000","last_undegraded":"2024-09-14T07:12:18.374205+0000","last_fullsized":"2024-09-14T07:12:18.374205+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:24.601874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5017,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369266+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:18.369266+0000","last_peered":"2024-09-14T07:12:18.369266+0000","last_clean":"2024-09-14T07:12:18.369266+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:18.369266+0000","last_undegraded":"2024-09-14T07:12:18.369266+0000","last_fullsized":"2024-09-14T07:12:18.369266+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82592,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289248,"statfs":{"total":96636764160,"available":96552189952,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607615,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70156,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301684,"statfs":{"total":96636764160,"available":96564924416,"internally_reserved":0,"allocated":2940928,"data_stored":1787422,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607616,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":2883584,"data_stored":1197054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607616,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":2860,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":2928640,"data_stored":1787441,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:24.965 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T07:12:24.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T07:12:25.299 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T07:12:25.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T07:12:25.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:25.619 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:25.637 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-14T07:12:25.280765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336728,"kb_used_data":17008,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894312,"statfs":{"total":579820584960,"available":579475775488,"internally_reserved":0,"allocated":17416192,"data_stored":8363136,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002270"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7019,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369738+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:18.369738+0000","last_peered":"2024-09-14T07:12:18.369738+0000","last_clean":"2024-09-14T07:12:18.369738+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:18.369738+0000","last_undegraded":"2024-09-14T07:12:18.369738+0000","last_fullsized":"2024-09-14T07:12:18.369738+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13054,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16746,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372658+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:18.372658+0000","last_peered":"2024-09-14T07:12:18.372658+0000","last_clean":"2024-09-14T07:12:18.372658+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:18.372658+0000","last_undegraded":"2024-09-14T07:12:18.372658+0000","last_fullsized":"2024-09-14T07:12:18.372658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20881,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372972+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:18.372972+0000","last_peered":"2024-09-14T07:12:18.372972+0000","last_clean":"2024-09-14T07:12:18.372972+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:18.372972+0000","last_undegraded":"2024-09-14T07:12:18.372972+0000","last_fullsized":"2024-09-14T07:12:18.372972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8737,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369079+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:18.369079+0000","last_peered":"2024-09-14T07:12:18.369079+0000","last_clean":"2024-09-14T07:12:18.369079+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:18.369079+0000","last_undegraded":"2024-09-14T07:12:18.369079+0000","last_fullsized":"2024-09-14T07:12:18.369079+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30507,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641648+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:18.641648+0000","last_peered":"2024-09-14T07:12:18.641648+0000","last_clean":"2024-09-14T07:12:18.641648+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.641648+0000","last_undegraded":"2024-09-14T07:12:18.641648+0000","last_fullsized":"2024-09-14T07:12:18.641648+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10673,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639203+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.639203+0000","last_peered":"2024-09-14T07:12:18.639203+0000","last_clean":"2024-09-14T07:12:18.639203+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.639203+0000","last_undegraded":"2024-09-14T07:12:18.639203+0000","last_fullsized":"2024-09-14T07:12:18.639203+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5487,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368465+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:18.368465+0000","last_peered":"2024-09-14T07:12:18.368465+0000","last_clean":"2024-09-14T07:12:18.368465+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:18.368465+0000","last_undegraded":"2024-09-14T07:12:18.368465+0000","last_fullsized":"2024-09-14T07:12:18.368465+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3844,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.376942+0000","last_change":"2024-09-14T07:12:18.376850+0000","last_active":"2024-09-14T07:12:18.376942+0000","last_peered":"2024-09-14T07:12:18.376942+0000","last_clean":"2024-09-14T07:12:18.376942+0000","last_became_active":"2024-09-14T07:12:18.374908+0000","last_became_peered":"2024-09-14T07:12:18.374908+0000","last_unstale":"2024-09-14T07:12:18.376942+0000","last_undegraded":"2024-09-14T07:12:18.376942+0000","last_fullsized":"2024-09-14T07:12:18.376942+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:14.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10084,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372949+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:18.372949+0000","last_peered":"2024-09-14T07:12:18.372949+0000","last_clean":"2024-09-14T07:12:18.372949+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:18.372949+0000","last_undegraded":"2024-09-14T07:12:18.372949+0000","last_fullsized":"2024-09-14T07:12:18.372949+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7144,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368999+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:18.368999+0000","last_peered":"2024-09-14T07:12:18.368999+0000","last_clean":"2024-09-14T07:12:18.368999+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:18.368999+0000","last_undegraded":"2024-09-14T07:12:18.368999+0000","last_fullsized":"2024-09-14T07:12:18.368999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368752+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:18.368752+0000","last_peered":"2024-09-14T07:12:18.368752+0000","last_clean":"2024-09-14T07:12:18.368752+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:18.368752+0000","last_undegraded":"2024-09-14T07:12:18.368752+0000","last_fullsized":"2024-09-14T07:12:18.368752+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369501+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:18.369501+0000","last_peered":"2024-09-14T07:12:18.369501+0000","last_clean":"2024-09-14T07:12:18.369501+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:18.369501+0000","last_undegraded":"2024-09-14T07:12:18.369501+0000","last_fullsized":"2024-09-14T07:12:18.369501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12875,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368991+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:18.368991+0000","last_peered":"2024-09-14T07:12:18.368991+0000","last_clean":"2024-09-14T07:12:18.368991+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:18.368991+0000","last_undegraded":"2024-09-14T07:12:18.368991+0000","last_fullsized":"2024-09-14T07:12:18.368991+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639779+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.639779+0000","last_peered":"2024-09-14T07:12:18.639779+0000","last_clean":"2024-09-14T07:12:18.639779+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.639779+0000","last_undegraded":"2024-09-14T07:12:18.639779+0000","last_fullsized":"2024-09-14T07:12:18.639779+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8377,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368716+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:18.368716+0000","last_peered":"2024-09-14T07:12:18.368716+0000","last_clean":"2024-09-14T07:12:18.368716+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:18.368716+0000","last_undegraded":"2024-09-14T07:12:18.368716+0000","last_fullsized":"2024-09-14T07:12:18.368716+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369005+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:18.369005+0000","last_peered":"2024-09-14T07:12:18.369005+0000","last_clean":"2024-09-14T07:12:18.369005+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:18.369005+0000","last_undegraded":"2024-09-14T07:12:18.369005+0000","last_fullsized":"2024-09-14T07:12:18.369005+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6739,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639507+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.639507+0000","last_peered":"2024-09-14T07:12:18.639507+0000","last_clean":"2024-09-14T07:12:18.639507+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.639507+0000","last_undegraded":"2024-09-14T07:12:18.639507+0000","last_fullsized":"2024-09-14T07:12:18.639507+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639605+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.639605+0000","last_peered":"2024-09-14T07:12:18.639605+0000","last_clean":"2024-09-14T07:12:18.639605+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.639605+0000","last_undegraded":"2024-09-14T07:12:18.639605+0000","last_fullsized":"2024-09-14T07:12:18.639605+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4101,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372761+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:18.372761+0000","last_peered":"2024-09-14T07:12:18.372761+0000","last_clean":"2024-09-14T07:12:18.372761+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:18.372761+0000","last_undegraded":"2024-09-14T07:12:18.372761+0000","last_fullsized":"2024-09-14T07:12:18.372761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10160,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641679+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:18.641679+0000","last_peered":"2024-09-14T07:12:18.641679+0000","last_clean":"2024-09-14T07:12:18.641679+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:18.641679+0000","last_undegraded":"2024-09-14T07:12:18.641679+0000","last_fullsized":"2024-09-14T07:12:18.641679+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":483,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372478+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:18.372478+0000","last_peered":"2024-09-14T07:12:18.372478+0000","last_clean":"2024-09-14T07:12:18.372478+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:18.372478+0000","last_undegraded":"2024-09-14T07:12:18.372478+0000","last_fullsized":"2024-09-14T07:12:18.372478+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.640028+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.640028+0000","last_peered":"2024-09-14T07:12:18.640028+0000","last_clean":"2024-09-14T07:12:18.640028+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.640028+0000","last_undegraded":"2024-09-14T07:12:18.640028+0000","last_fullsized":"2024-09-14T07:12:18.640028+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":290,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368448+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:18.368448+0000","last_peered":"2024-09-14T07:12:18.368448+0000","last_clean":"2024-09-14T07:12:18.368448+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:18.368448+0000","last_undegraded":"2024-09-14T07:12:18.368448+0000","last_fullsized":"2024-09-14T07:12:18.368448+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13050,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368577+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:18.368577+0000","last_peered":"2024-09-14T07:12:18.368577+0000","last_clean":"2024-09-14T07:12:18.368577+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:18.368577+0000","last_undegraded":"2024-09-14T07:12:18.368577+0000","last_fullsized":"2024-09-14T07:12:18.368577+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368438+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:18.368438+0000","last_peered":"2024-09-14T07:12:18.368438+0000","last_clean":"2024-09-14T07:12:18.368438+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:18.368438+0000","last_undegraded":"2024-09-14T07:12:18.368438+0000","last_fullsized":"2024-09-14T07:12:18.368438+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7918,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639383+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.639383+0000","last_peered":"2024-09-14T07:12:18.639383+0000","last_clean":"2024-09-14T07:12:18.639383+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.639383+0000","last_undegraded":"2024-09-14T07:12:18.639383+0000","last_fullsized":"2024-09-14T07:12:18.639383+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372692+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:18.372692+0000","last_peered":"2024-09-14T07:12:18.372692+0000","last_clean":"2024-09-14T07:12:18.372692+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:18.372692+0000","last_undegraded":"2024-09-14T07:12:18.372692+0000","last_fullsized":"2024-09-14T07:12:18.372692+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368746+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:18.368746+0000","last_peered":"2024-09-14T07:12:18.368746+0000","last_clean":"2024-09-14T07:12:18.368746+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:18.368746+0000","last_undegraded":"2024-09-14T07:12:18.368746+0000","last_fullsized":"2024-09-14T07:12:18.368746+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10308,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8434,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641712+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:18.641712+0000","last_peered":"2024-09-14T07:12:18.641712+0000","last_clean":"2024-09-14T07:12:18.641712+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:18.641712+0000","last_undegraded":"2024-09-14T07:12:18.641712+0000","last_fullsized":"2024-09-14T07:12:18.641712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8268,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368658+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:18.368658+0000","last_peered":"2024-09-14T07:12:18.368658+0000","last_clean":"2024-09-14T07:12:18.368658+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:18.368658+0000","last_undegraded":"2024-09-14T07:12:18.368658+0000","last_fullsized":"2024-09-14T07:12:18.368658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8181,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639165+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.639165+0000","last_peered":"2024-09-14T07:12:18.639165+0000","last_clean":"2024-09-14T07:12:18.639165+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.639165+0000","last_undegraded":"2024-09-14T07:12:18.639165+0000","last_fullsized":"2024-09-14T07:12:18.639165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15528,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372744+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:18.372744+0000","last_peered":"2024-09-14T07:12:18.372744+0000","last_clean":"2024-09-14T07:12:18.372744+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.372744+0000","last_undegraded":"2024-09-14T07:12:18.372744+0000","last_fullsized":"2024-09-14T07:12:18.372744+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368844+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:18.368844+0000","last_peered":"2024-09-14T07:12:18.368844+0000","last_clean":"2024-09-14T07:12:18.368844+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:18.368844+0000","last_undegraded":"2024-09-14T07:12:18.368844+0000","last_fullsized":"2024-09-14T07:12:18.368844+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17500,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373074+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:18.373074+0000","last_peered":"2024-09-14T07:12:18.373074+0000","last_clean":"2024-09-14T07:12:18.373074+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:18.373074+0000","last_undegraded":"2024-09-14T07:12:18.373074+0000","last_fullsized":"2024-09-14T07:12:18.373074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8588,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373393+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:18.373393+0000","last_peered":"2024-09-14T07:12:18.373393+0000","last_clean":"2024-09-14T07:12:18.373393+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:18.373393+0000","last_undegraded":"2024-09-14T07:12:18.373393+0000","last_fullsized":"2024-09-14T07:12:18.373393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8201,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372420+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:18.372420+0000","last_peered":"2024-09-14T07:12:18.372420+0000","last_clean":"2024-09-14T07:12:18.372420+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:18.372420+0000","last_undegraded":"2024-09-14T07:12:18.372420+0000","last_fullsized":"2024-09-14T07:12:18.372420+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368461+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:18.368461+0000","last_peered":"2024-09-14T07:12:18.368461+0000","last_clean":"2024-09-14T07:12:18.368461+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:18.368461+0000","last_undegraded":"2024-09-14T07:12:18.368461+0000","last_fullsized":"2024-09-14T07:12:18.368461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7337,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.374205+0000","last_change":"2024-09-14T07:12:18.374075+0000","last_active":"2024-09-14T07:12:18.374205+0000","last_peered":"2024-09-14T07:12:18.374205+0000","last_clean":"2024-09-14T07:12:18.374205+0000","last_became_active":"2024-09-14T07:12:18.371429+0000","last_became_peered":"2024-09-14T07:12:18.371429+0000","last_unstale":"2024-09-14T07:12:18.374205+0000","last_undegraded":"2024-09-14T07:12:18.374205+0000","last_fullsized":"2024-09-14T07:12:18.374205+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:24.601874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5017,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369266+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:18.369266+0000","last_peered":"2024-09-14T07:12:18.369266+0000","last_clean":"2024-09-14T07:12:18.369266+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:18.369266+0000","last_undegraded":"2024-09-14T07:12:18.369266+0000","last_fullsized":"2024-09-14T07:12:18.369266+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82592,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289248,"statfs":{"total":96636764160,"available":96552189952,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607615,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70156,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301684,"statfs":{"total":96636764160,"available":96564924416,"internally_reserved":0,"allocated":2940928,"data_stored":1787422,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607616,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":2883584,"data_stored":1197054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607616,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":2860,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":2928640,"data_stored":1787441,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:25.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:25.929 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:25.947 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-14T07:12:25.280765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336728,"kb_used_data":17008,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894312,"statfs":{"total":579820584960,"available":579475775488,"internally_reserved":0,"allocated":17416192,"data_stored":8363136,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002270"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7019,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369738+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:18.369738+0000","last_peered":"2024-09-14T07:12:18.369738+0000","last_clean":"2024-09-14T07:12:18.369738+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:18.369738+0000","last_undegraded":"2024-09-14T07:12:18.369738+0000","last_fullsized":"2024-09-14T07:12:18.369738+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13054,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16746,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372658+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:18.372658+0000","last_peered":"2024-09-14T07:12:18.372658+0000","last_clean":"2024-09-14T07:12:18.372658+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:18.372658+0000","last_undegraded":"2024-09-14T07:12:18.372658+0000","last_fullsized":"2024-09-14T07:12:18.372658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20881,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372972+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:18.372972+0000","last_peered":"2024-09-14T07:12:18.372972+0000","last_clean":"2024-09-14T07:12:18.372972+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:18.372972+0000","last_undegraded":"2024-09-14T07:12:18.372972+0000","last_fullsized":"2024-09-14T07:12:18.372972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8737,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369079+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:18.369079+0000","last_peered":"2024-09-14T07:12:18.369079+0000","last_clean":"2024-09-14T07:12:18.369079+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:18.369079+0000","last_undegraded":"2024-09-14T07:12:18.369079+0000","last_fullsized":"2024-09-14T07:12:18.369079+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30507,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641648+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:18.641648+0000","last_peered":"2024-09-14T07:12:18.641648+0000","last_clean":"2024-09-14T07:12:18.641648+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.641648+0000","last_undegraded":"2024-09-14T07:12:18.641648+0000","last_fullsized":"2024-09-14T07:12:18.641648+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10673,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639203+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.639203+0000","last_peered":"2024-09-14T07:12:18.639203+0000","last_clean":"2024-09-14T07:12:18.639203+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.639203+0000","last_undegraded":"2024-09-14T07:12:18.639203+0000","last_fullsized":"2024-09-14T07:12:18.639203+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5487,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368465+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:18.368465+0000","last_peered":"2024-09-14T07:12:18.368465+0000","last_clean":"2024-09-14T07:12:18.368465+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:18.368465+0000","last_undegraded":"2024-09-14T07:12:18.368465+0000","last_fullsized":"2024-09-14T07:12:18.368465+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3844,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.376942+0000","last_change":"2024-09-14T07:12:18.376850+0000","last_active":"2024-09-14T07:12:18.376942+0000","last_peered":"2024-09-14T07:12:18.376942+0000","last_clean":"2024-09-14T07:12:18.376942+0000","last_became_active":"2024-09-14T07:12:18.374908+0000","last_became_peered":"2024-09-14T07:12:18.374908+0000","last_unstale":"2024-09-14T07:12:18.376942+0000","last_undegraded":"2024-09-14T07:12:18.376942+0000","last_fullsized":"2024-09-14T07:12:18.376942+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:14.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10084,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372949+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:18.372949+0000","last_peered":"2024-09-14T07:12:18.372949+0000","last_clean":"2024-09-14T07:12:18.372949+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:18.372949+0000","last_undegraded":"2024-09-14T07:12:18.372949+0000","last_fullsized":"2024-09-14T07:12:18.372949+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7144,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368999+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:18.368999+0000","last_peered":"2024-09-14T07:12:18.368999+0000","last_clean":"2024-09-14T07:12:18.368999+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:18.368999+0000","last_undegraded":"2024-09-14T07:12:18.368999+0000","last_fullsized":"2024-09-14T07:12:18.368999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368752+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:18.368752+0000","last_peered":"2024-09-14T07:12:18.368752+0000","last_clean":"2024-09-14T07:12:18.368752+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:18.368752+0000","last_undegraded":"2024-09-14T07:12:18.368752+0000","last_fullsized":"2024-09-14T07:12:18.368752+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369501+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:18.369501+0000","last_peered":"2024-09-14T07:12:18.369501+0000","last_clean":"2024-09-14T07:12:18.369501+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:18.369501+0000","last_undegraded":"2024-09-14T07:12:18.369501+0000","last_fullsized":"2024-09-14T07:12:18.369501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12875,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368991+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:18.368991+0000","last_peered":"2024-09-14T07:12:18.368991+0000","last_clean":"2024-09-14T07:12:18.368991+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:18.368991+0000","last_undegraded":"2024-09-14T07:12:18.368991+0000","last_fullsized":"2024-09-14T07:12:18.368991+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639779+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.639779+0000","last_peered":"2024-09-14T07:12:18.639779+0000","last_clean":"2024-09-14T07:12:18.639779+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.639779+0000","last_undegraded":"2024-09-14T07:12:18.639779+0000","last_fullsized":"2024-09-14T07:12:18.639779+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8377,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368716+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:18.368716+0000","last_peered":"2024-09-14T07:12:18.368716+0000","last_clean":"2024-09-14T07:12:18.368716+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:18.368716+0000","last_undegraded":"2024-09-14T07:12:18.368716+0000","last_fullsized":"2024-09-14T07:12:18.368716+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369005+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:18.369005+0000","last_peered":"2024-09-14T07:12:18.369005+0000","last_clean":"2024-09-14T07:12:18.369005+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:18.369005+0000","last_undegraded":"2024-09-14T07:12:18.369005+0000","last_fullsized":"2024-09-14T07:12:18.369005+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6739,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639507+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.639507+0000","last_peered":"2024-09-14T07:12:18.639507+0000","last_clean":"2024-09-14T07:12:18.639507+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.639507+0000","last_undegraded":"2024-09-14T07:12:18.639507+0000","last_fullsized":"2024-09-14T07:12:18.639507+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639605+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.639605+0000","last_peered":"2024-09-14T07:12:18.639605+0000","last_clean":"2024-09-14T07:12:18.639605+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.639605+0000","last_undegraded":"2024-09-14T07:12:18.639605+0000","last_fullsized":"2024-09-14T07:12:18.639605+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4101,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372761+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:18.372761+0000","last_peered":"2024-09-14T07:12:18.372761+0000","last_clean":"2024-09-14T07:12:18.372761+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:18.372761+0000","last_undegraded":"2024-09-14T07:12:18.372761+0000","last_fullsized":"2024-09-14T07:12:18.372761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10160,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641679+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:18.641679+0000","last_peered":"2024-09-14T07:12:18.641679+0000","last_clean":"2024-09-14T07:12:18.641679+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:18.641679+0000","last_undegraded":"2024-09-14T07:12:18.641679+0000","last_fullsized":"2024-09-14T07:12:18.641679+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":483,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372478+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:18.372478+0000","last_peered":"2024-09-14T07:12:18.372478+0000","last_clean":"2024-09-14T07:12:18.372478+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:18.372478+0000","last_undegraded":"2024-09-14T07:12:18.372478+0000","last_fullsized":"2024-09-14T07:12:18.372478+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.640028+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.640028+0000","last_peered":"2024-09-14T07:12:18.640028+0000","last_clean":"2024-09-14T07:12:18.640028+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.640028+0000","last_undegraded":"2024-09-14T07:12:18.640028+0000","last_fullsized":"2024-09-14T07:12:18.640028+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":290,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368448+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:18.368448+0000","last_peered":"2024-09-14T07:12:18.368448+0000","last_clean":"2024-09-14T07:12:18.368448+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:18.368448+0000","last_undegraded":"2024-09-14T07:12:18.368448+0000","last_fullsized":"2024-09-14T07:12:18.368448+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13050,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368577+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:18.368577+0000","last_peered":"2024-09-14T07:12:18.368577+0000","last_clean":"2024-09-14T07:12:18.368577+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:18.368577+0000","last_undegraded":"2024-09-14T07:12:18.368577+0000","last_fullsized":"2024-09-14T07:12:18.368577+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368438+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:18.368438+0000","last_peered":"2024-09-14T07:12:18.368438+0000","last_clean":"2024-09-14T07:12:18.368438+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:18.368438+0000","last_undegraded":"2024-09-14T07:12:18.368438+0000","last_fullsized":"2024-09-14T07:12:18.368438+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7918,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639383+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.639383+0000","last_peered":"2024-09-14T07:12:18.639383+0000","last_clean":"2024-09-14T07:12:18.639383+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.639383+0000","last_undegraded":"2024-09-14T07:12:18.639383+0000","last_fullsized":"2024-09-14T07:12:18.639383+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372692+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:18.372692+0000","last_peered":"2024-09-14T07:12:18.372692+0000","last_clean":"2024-09-14T07:12:18.372692+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:18.372692+0000","last_undegraded":"2024-09-14T07:12:18.372692+0000","last_fullsized":"2024-09-14T07:12:18.372692+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368746+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:18.368746+0000","last_peered":"2024-09-14T07:12:18.368746+0000","last_clean":"2024-09-14T07:12:18.368746+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:18.368746+0000","last_undegraded":"2024-09-14T07:12:18.368746+0000","last_fullsized":"2024-09-14T07:12:18.368746+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10308,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8434,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641712+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:18.641712+0000","last_peered":"2024-09-14T07:12:18.641712+0000","last_clean":"2024-09-14T07:12:18.641712+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:18.641712+0000","last_undegraded":"2024-09-14T07:12:18.641712+0000","last_fullsized":"2024-09-14T07:12:18.641712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8268,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368658+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:18.368658+0000","last_peered":"2024-09-14T07:12:18.368658+0000","last_clean":"2024-09-14T07:12:18.368658+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:18.368658+0000","last_undegraded":"2024-09-14T07:12:18.368658+0000","last_fullsized":"2024-09-14T07:12:18.368658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8181,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639165+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.639165+0000","last_peered":"2024-09-14T07:12:18.639165+0000","last_clean":"2024-09-14T07:12:18.639165+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.639165+0000","last_undegraded":"2024-09-14T07:12:18.639165+0000","last_fullsized":"2024-09-14T07:12:18.639165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15528,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372744+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:18.372744+0000","last_peered":"2024-09-14T07:12:18.372744+0000","last_clean":"2024-09-14T07:12:18.372744+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.372744+0000","last_undegraded":"2024-09-14T07:12:18.372744+0000","last_fullsized":"2024-09-14T07:12:18.372744+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368844+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:18.368844+0000","last_peered":"2024-09-14T07:12:18.368844+0000","last_clean":"2024-09-14T07:12:18.368844+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:18.368844+0000","last_undegraded":"2024-09-14T07:12:18.368844+0000","last_fullsized":"2024-09-14T07:12:18.368844+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17500,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373074+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:18.373074+0000","last_peered":"2024-09-14T07:12:18.373074+0000","last_clean":"2024-09-14T07:12:18.373074+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:18.373074+0000","last_undegraded":"2024-09-14T07:12:18.373074+0000","last_fullsized":"2024-09-14T07:12:18.373074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8588,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373393+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:18.373393+0000","last_peered":"2024-09-14T07:12:18.373393+0000","last_clean":"2024-09-14T07:12:18.373393+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:18.373393+0000","last_undegraded":"2024-09-14T07:12:18.373393+0000","last_fullsized":"2024-09-14T07:12:18.373393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8201,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372420+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:18.372420+0000","last_peered":"2024-09-14T07:12:18.372420+0000","last_clean":"2024-09-14T07:12:18.372420+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:18.372420+0000","last_undegraded":"2024-09-14T07:12:18.372420+0000","last_fullsized":"2024-09-14T07:12:18.372420+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368461+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:18.368461+0000","last_peered":"2024-09-14T07:12:18.368461+0000","last_clean":"2024-09-14T07:12:18.368461+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:18.368461+0000","last_undegraded":"2024-09-14T07:12:18.368461+0000","last_fullsized":"2024-09-14T07:12:18.368461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7337,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.374205+0000","last_change":"2024-09-14T07:12:18.374075+0000","last_active":"2024-09-14T07:12:18.374205+0000","last_peered":"2024-09-14T07:12:18.374205+0000","last_clean":"2024-09-14T07:12:18.374205+0000","last_became_active":"2024-09-14T07:12:18.371429+0000","last_became_peered":"2024-09-14T07:12:18.371429+0000","last_unstale":"2024-09-14T07:12:18.374205+0000","last_undegraded":"2024-09-14T07:12:18.374205+0000","last_fullsized":"2024-09-14T07:12:18.374205+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:24.601874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5017,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369266+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:18.369266+0000","last_peered":"2024-09-14T07:12:18.369266+0000","last_clean":"2024-09-14T07:12:18.369266+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:18.369266+0000","last_undegraded":"2024-09-14T07:12:18.369266+0000","last_fullsized":"2024-09-14T07:12:18.369266+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82592,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289248,"statfs":{"total":96636764160,"available":96552189952,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607615,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70156,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301684,"statfs":{"total":96636764160,"available":96564924416,"internally_reserved":0,"allocated":2940928,"data_stored":1787422,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607616,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":2883584,"data_stored":1197054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607616,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":2860,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":2928640,"data_stored":1787441,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:25.949 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T07:12:25.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:26.239 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:26.240 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:26.257 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-14T07:12:25.280765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336728,"kb_used_data":17008,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894312,"statfs":{"total":579820584960,"available":579475775488,"internally_reserved":0,"allocated":17416192,"data_stored":8363136,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002270"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7019,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369738+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:18.369738+0000","last_peered":"2024-09-14T07:12:18.369738+0000","last_clean":"2024-09-14T07:12:18.369738+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:18.369738+0000","last_undegraded":"2024-09-14T07:12:18.369738+0000","last_fullsized":"2024-09-14T07:12:18.369738+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13054,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16746,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372658+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:18.372658+0000","last_peered":"2024-09-14T07:12:18.372658+0000","last_clean":"2024-09-14T07:12:18.372658+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:18.372658+0000","last_undegraded":"2024-09-14T07:12:18.372658+0000","last_fullsized":"2024-09-14T07:12:18.372658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20881,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372972+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:18.372972+0000","last_peered":"2024-09-14T07:12:18.372972+0000","last_clean":"2024-09-14T07:12:18.372972+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:18.372972+0000","last_undegraded":"2024-09-14T07:12:18.372972+0000","last_fullsized":"2024-09-14T07:12:18.372972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8737,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369079+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:18.369079+0000","last_peered":"2024-09-14T07:12:18.369079+0000","last_clean":"2024-09-14T07:12:18.369079+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:18.369079+0000","last_undegraded":"2024-09-14T07:12:18.369079+0000","last_fullsized":"2024-09-14T07:12:18.369079+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30507,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641648+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:18.641648+0000","last_peered":"2024-09-14T07:12:18.641648+0000","last_clean":"2024-09-14T07:12:18.641648+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.641648+0000","last_undegraded":"2024-09-14T07:12:18.641648+0000","last_fullsized":"2024-09-14T07:12:18.641648+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10673,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639203+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.639203+0000","last_peered":"2024-09-14T07:12:18.639203+0000","last_clean":"2024-09-14T07:12:18.639203+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.639203+0000","last_undegraded":"2024-09-14T07:12:18.639203+0000","last_fullsized":"2024-09-14T07:12:18.639203+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5487,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368465+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:18.368465+0000","last_peered":"2024-09-14T07:12:18.368465+0000","last_clean":"2024-09-14T07:12:18.368465+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:18.368465+0000","last_undegraded":"2024-09-14T07:12:18.368465+0000","last_fullsized":"2024-09-14T07:12:18.368465+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3844,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.376942+0000","last_change":"2024-09-14T07:12:18.376850+0000","last_active":"2024-09-14T07:12:18.376942+0000","last_peered":"2024-09-14T07:12:18.376942+0000","last_clean":"2024-09-14T07:12:18.376942+0000","last_became_active":"2024-09-14T07:12:18.374908+0000","last_became_peered":"2024-09-14T07:12:18.374908+0000","last_unstale":"2024-09-14T07:12:18.376942+0000","last_undegraded":"2024-09-14T07:12:18.376942+0000","last_fullsized":"2024-09-14T07:12:18.376942+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:14.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10084,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372949+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:18.372949+0000","last_peered":"2024-09-14T07:12:18.372949+0000","last_clean":"2024-09-14T07:12:18.372949+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:18.372949+0000","last_undegraded":"2024-09-14T07:12:18.372949+0000","last_fullsized":"2024-09-14T07:12:18.372949+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7144,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368999+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:18.368999+0000","last_peered":"2024-09-14T07:12:18.368999+0000","last_clean":"2024-09-14T07:12:18.368999+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:18.368999+0000","last_undegraded":"2024-09-14T07:12:18.368999+0000","last_fullsized":"2024-09-14T07:12:18.368999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368752+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:18.368752+0000","last_peered":"2024-09-14T07:12:18.368752+0000","last_clean":"2024-09-14T07:12:18.368752+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:18.368752+0000","last_undegraded":"2024-09-14T07:12:18.368752+0000","last_fullsized":"2024-09-14T07:12:18.368752+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369501+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:18.369501+0000","last_peered":"2024-09-14T07:12:18.369501+0000","last_clean":"2024-09-14T07:12:18.369501+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:18.369501+0000","last_undegraded":"2024-09-14T07:12:18.369501+0000","last_fullsized":"2024-09-14T07:12:18.369501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12875,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368991+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:18.368991+0000","last_peered":"2024-09-14T07:12:18.368991+0000","last_clean":"2024-09-14T07:12:18.368991+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:18.368991+0000","last_undegraded":"2024-09-14T07:12:18.368991+0000","last_fullsized":"2024-09-14T07:12:18.368991+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639779+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.639779+0000","last_peered":"2024-09-14T07:12:18.639779+0000","last_clean":"2024-09-14T07:12:18.639779+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.639779+0000","last_undegraded":"2024-09-14T07:12:18.639779+0000","last_fullsized":"2024-09-14T07:12:18.639779+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8377,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368716+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:18.368716+0000","last_peered":"2024-09-14T07:12:18.368716+0000","last_clean":"2024-09-14T07:12:18.368716+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:18.368716+0000","last_undegraded":"2024-09-14T07:12:18.368716+0000","last_fullsized":"2024-09-14T07:12:18.368716+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369005+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:18.369005+0000","last_peered":"2024-09-14T07:12:18.369005+0000","last_clean":"2024-09-14T07:12:18.369005+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:18.369005+0000","last_undegraded":"2024-09-14T07:12:18.369005+0000","last_fullsized":"2024-09-14T07:12:18.369005+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6739,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639507+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.639507+0000","last_peered":"2024-09-14T07:12:18.639507+0000","last_clean":"2024-09-14T07:12:18.639507+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.639507+0000","last_undegraded":"2024-09-14T07:12:18.639507+0000","last_fullsized":"2024-09-14T07:12:18.639507+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639605+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.639605+0000","last_peered":"2024-09-14T07:12:18.639605+0000","last_clean":"2024-09-14T07:12:18.639605+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.639605+0000","last_undegraded":"2024-09-14T07:12:18.639605+0000","last_fullsized":"2024-09-14T07:12:18.639605+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4101,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372761+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:18.372761+0000","last_peered":"2024-09-14T07:12:18.372761+0000","last_clean":"2024-09-14T07:12:18.372761+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:18.372761+0000","last_undegraded":"2024-09-14T07:12:18.372761+0000","last_fullsized":"2024-09-14T07:12:18.372761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10160,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641679+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:18.641679+0000","last_peered":"2024-09-14T07:12:18.641679+0000","last_clean":"2024-09-14T07:12:18.641679+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:18.641679+0000","last_undegraded":"2024-09-14T07:12:18.641679+0000","last_fullsized":"2024-09-14T07:12:18.641679+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":483,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372478+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:18.372478+0000","last_peered":"2024-09-14T07:12:18.372478+0000","last_clean":"2024-09-14T07:12:18.372478+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:18.372478+0000","last_undegraded":"2024-09-14T07:12:18.372478+0000","last_fullsized":"2024-09-14T07:12:18.372478+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.640028+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.640028+0000","last_peered":"2024-09-14T07:12:18.640028+0000","last_clean":"2024-09-14T07:12:18.640028+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.640028+0000","last_undegraded":"2024-09-14T07:12:18.640028+0000","last_fullsized":"2024-09-14T07:12:18.640028+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":290,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368448+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:18.368448+0000","last_peered":"2024-09-14T07:12:18.368448+0000","last_clean":"2024-09-14T07:12:18.368448+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:18.368448+0000","last_undegraded":"2024-09-14T07:12:18.368448+0000","last_fullsized":"2024-09-14T07:12:18.368448+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13050,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368577+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:18.368577+0000","last_peered":"2024-09-14T07:12:18.368577+0000","last_clean":"2024-09-14T07:12:18.368577+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:18.368577+0000","last_undegraded":"2024-09-14T07:12:18.368577+0000","last_fullsized":"2024-09-14T07:12:18.368577+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368438+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:18.368438+0000","last_peered":"2024-09-14T07:12:18.368438+0000","last_clean":"2024-09-14T07:12:18.368438+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:18.368438+0000","last_undegraded":"2024-09-14T07:12:18.368438+0000","last_fullsized":"2024-09-14T07:12:18.368438+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7918,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639383+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.639383+0000","last_peered":"2024-09-14T07:12:18.639383+0000","last_clean":"2024-09-14T07:12:18.639383+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.639383+0000","last_undegraded":"2024-09-14T07:12:18.639383+0000","last_fullsized":"2024-09-14T07:12:18.639383+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372692+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:18.372692+0000","last_peered":"2024-09-14T07:12:18.372692+0000","last_clean":"2024-09-14T07:12:18.372692+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:18.372692+0000","last_undegraded":"2024-09-14T07:12:18.372692+0000","last_fullsized":"2024-09-14T07:12:18.372692+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368746+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:18.368746+0000","last_peered":"2024-09-14T07:12:18.368746+0000","last_clean":"2024-09-14T07:12:18.368746+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:18.368746+0000","last_undegraded":"2024-09-14T07:12:18.368746+0000","last_fullsized":"2024-09-14T07:12:18.368746+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10308,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8434,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641712+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:18.641712+0000","last_peered":"2024-09-14T07:12:18.641712+0000","last_clean":"2024-09-14T07:12:18.641712+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:18.641712+0000","last_undegraded":"2024-09-14T07:12:18.641712+0000","last_fullsized":"2024-09-14T07:12:18.641712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8268,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368658+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:18.368658+0000","last_peered":"2024-09-14T07:12:18.368658+0000","last_clean":"2024-09-14T07:12:18.368658+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:18.368658+0000","last_undegraded":"2024-09-14T07:12:18.368658+0000","last_fullsized":"2024-09-14T07:12:18.368658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8181,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639165+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.639165+0000","last_peered":"2024-09-14T07:12:18.639165+0000","last_clean":"2024-09-14T07:12:18.639165+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.639165+0000","last_undegraded":"2024-09-14T07:12:18.639165+0000","last_fullsized":"2024-09-14T07:12:18.639165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15528,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372744+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:18.372744+0000","last_peered":"2024-09-14T07:12:18.372744+0000","last_clean":"2024-09-14T07:12:18.372744+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.372744+0000","last_undegraded":"2024-09-14T07:12:18.372744+0000","last_fullsized":"2024-09-14T07:12:18.372744+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368844+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:18.368844+0000","last_peered":"2024-09-14T07:12:18.368844+0000","last_clean":"2024-09-14T07:12:18.368844+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:18.368844+0000","last_undegraded":"2024-09-14T07:12:18.368844+0000","last_fullsized":"2024-09-14T07:12:18.368844+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17500,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373074+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:18.373074+0000","last_peered":"2024-09-14T07:12:18.373074+0000","last_clean":"2024-09-14T07:12:18.373074+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:18.373074+0000","last_undegraded":"2024-09-14T07:12:18.373074+0000","last_fullsized":"2024-09-14T07:12:18.373074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8588,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373393+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:18.373393+0000","last_peered":"2024-09-14T07:12:18.373393+0000","last_clean":"2024-09-14T07:12:18.373393+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:18.373393+0000","last_undegraded":"2024-09-14T07:12:18.373393+0000","last_fullsized":"2024-09-14T07:12:18.373393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8201,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372420+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:18.372420+0000","last_peered":"2024-09-14T07:12:18.372420+0000","last_clean":"2024-09-14T07:12:18.372420+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:18.372420+0000","last_undegraded":"2024-09-14T07:12:18.372420+0000","last_fullsized":"2024-09-14T07:12:18.372420+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368461+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:18.368461+0000","last_peered":"2024-09-14T07:12:18.368461+0000","last_clean":"2024-09-14T07:12:18.368461+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:18.368461+0000","last_undegraded":"2024-09-14T07:12:18.368461+0000","last_fullsized":"2024-09-14T07:12:18.368461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7337,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.374205+0000","last_change":"2024-09-14T07:12:18.374075+0000","last_active":"2024-09-14T07:12:18.374205+0000","last_peered":"2024-09-14T07:12:18.374205+0000","last_clean":"2024-09-14T07:12:18.374205+0000","last_became_active":"2024-09-14T07:12:18.371429+0000","last_became_peered":"2024-09-14T07:12:18.371429+0000","last_unstale":"2024-09-14T07:12:18.374205+0000","last_undegraded":"2024-09-14T07:12:18.374205+0000","last_fullsized":"2024-09-14T07:12:18.374205+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:24.601874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5017,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369266+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:18.369266+0000","last_peered":"2024-09-14T07:12:18.369266+0000","last_clean":"2024-09-14T07:12:18.369266+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:18.369266+0000","last_undegraded":"2024-09-14T07:12:18.369266+0000","last_fullsized":"2024-09-14T07:12:18.369266+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82592,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289248,"statfs":{"total":96636764160,"available":96552189952,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607615,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607615,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70156,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301684,"statfs":{"total":96636764160,"available":96564924416,"internally_reserved":0,"allocated":2940928,"data_stored":1787422,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607616,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":2883584,"data_stored":1197054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607616,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":2860,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":2928640,"data_stored":1787441,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:26.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:12:26.557 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:26.558 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":239,"fsid":"717e3012-acdf-42c1-a4e0-d4c94afd9f15","created":"2024-09-14T07:07:18.019909+0000","modified":"2024-09-14T07:12:18.361824+0000","last_up_change":"2024-09-14T07:11:42.075125+0000","last_in_change":"2024-09-14T07:10:52.180916+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T07:07:31.246122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T07:07:34.989638+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T07:07:45.828508+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"223","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":46,"snap_epoch":223,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c97eae7-11c5-4962-a62d-04d698e50114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":182,"up_from":209,"up_thru":238,"down_at":184,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6817","nonce":2587782519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6819","nonce":2587782519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6823","nonce":2587782519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2587782519},{"type":"v1","addr":"172.21.15.5:6821","nonce":2587782519}]},"public_addr":"172.21.15.5:6817/2587782519","cluster_addr":"172.21.15.5:6819/2587782519","heartbeat_back_addr":"172.21.15.5:6823/2587782519","heartbeat_front_addr":"172.21.15.5:6821/2587782519","state":["exists","up"]},{"osd":1,"uuid":"91d33ac7-8429-4237-bb93-1259b318642a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6801","nonce":1704459401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6803","nonce":1704459401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6807","nonce":1704459401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1704459401},{"type":"v1","addr":"172.21.15.5:6805","nonce":1704459401}]},"public_addr":"172.21.15.5:6801/1704459401","cluster_addr":"172.21.15.5:6803/1704459401","heartbeat_back_addr":"172.21.15.5:6807/1704459401","heartbeat_front_addr":"172.21.15.5:6805/1704459401","state":["exists","up"]},{"osd":2,"uuid":"02202e5a-e516-4438-9378-7fad80297e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":230,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6809","nonce":2565128645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6811","nonce":2565128645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6815","nonce":2565128645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2565128645},{"type":"v1","addr":"172.21.15.5:6813","nonce":2565128645}]},"public_addr":"172.21.15.5:6809/2565128645","cluster_addr":"172.21.15.5:6811/2565128645","heartbeat_back_addr":"172.21.15.5:6815/2565128645","heartbeat_front_addr":"172.21.15.5:6813/2565128645","state":["exists","up"]},{"osd":3,"uuid":"57a45940-90af-484d-9570-8de1c6b2252b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":209,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6801","nonce":2226792373}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6803","nonce":2226792373}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6807","nonce":2226792373}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":2226792373},{"type":"v1","addr":"172.21.15.110:6805","nonce":2226792373}]},"public_addr":"172.21.15.110:6801/2226792373","cluster_addr":"172.21.15.110:6803/2226792373","heartbeat_back_addr":"172.21.15.110:6807/2226792373","heartbeat_front_addr":"172.21.15.110:6805/2226792373","state":["exists","up"]},{"osd":4,"uuid":"8150dd96-bfa0-404f-ae37-f18897bcc93e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":212,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6817","nonce":940255113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6819","nonce":940255113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6823","nonce":940255113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":940255113},{"type":"v1","addr":"172.21.15.110:6821","nonce":940255113}]},"public_addr":"172.21.15.110:6817/940255113","cluster_addr":"172.21.15.110:6819/940255113","heartbeat_back_addr":"172.21.15.110:6823/940255113","heartbeat_front_addr":"172.21.15.110:6821/940255113","state":["exists","up"]},{"osd":5,"uuid":"59d63d5d-8ef9-43b2-8d7f-49e8e8385435","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":217,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6809","nonce":4012173203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6811","nonce":4012173203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6815","nonce":4012173203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4012173203},{"type":"v1","addr":"172.21.15.110:6813","nonce":4012173203}]},"public_addr":"172.21.15.110:6809/4012173203","cluster_addr":"172.21.15.110:6811/4012173203","heartbeat_back_addr":"172.21.15.110:6815/4012173203","heartbeat_front_addr":"172.21.15.110:6813/4012173203","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.952391+0000","dead_epoch":184},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.101881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.869317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.961339+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:26.296733+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T07:07:25.723820+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1a","osds":[4,0]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:12:27.575 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T07:12:27.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:12:27.735 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:12:27.735 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:12:27.735 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:12:27.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T07:12:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T07:12:28.063 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T07:12:28.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:12:28.202 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:12:28.202 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:12:28.202 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:12:28.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T07:12:28.526 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T07:12:28.546 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T07:12:28.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:12:28.685 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:12:28.685 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:12:28.686 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:12:28.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T07:12:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T07:12:29.011 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T07:12:29.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:12:29.161 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:12:29.161 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:12:29.161 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:12:29.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T07:12:29.468 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T07:12:29.485 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T07:12:29.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:12:29.638 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:12:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:12:29.638 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:12:29.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T07:12:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T07:12:29.972 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T07:12:29.972 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:12:30.119 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-14T07:12:30.119 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:12:30.120 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-14T07:12:30.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T07:12:30.434 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T07:12:30.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:30.752 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:30.768 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-14T07:12:29.281978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19582,"ondisk_log_size":19582,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":336728,"kb_used_data":17008,"kb_used_omap":35,"kb_used_meta":319516,"kb_avail":565894312,"statfs":{"total":579820584960,"available":579475775488,"internally_reserved":0,"allocated":17416192,"data_stored":8363136,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":327184919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.917202"},"pg_stats":[{"pgid":"3.1f","version":"223'382","reported_seq":7019,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369738+0000","last_change":"2024-09-14T07:11:55.907742+0000","last_active":"2024-09-14T07:12:18.369738+0000","last_peered":"2024-09-14T07:12:18.369738+0000","last_clean":"2024-09-14T07:12:18.369738+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:18.369738+0000","last_undegraded":"2024-09-14T07:12:18.369738+0000","last_fullsized":"2024-09-14T07:12:18.369738+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:45:37.645194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"223'735","reported_seq":13054,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.304849+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:08:13.717702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"223'1629","reported_seq":16746,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372658+0000","last_change":"2024-09-14T07:11:55.862453+0000","last_active":"2024-09-14T07:12:18.372658+0000","last_peered":"2024-09-14T07:12:18.372658+0000","last_clean":"2024-09-14T07:12:18.372658+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:18.372658+0000","last_undegraded":"2024-09-14T07:12:18.372658+0000","last_fullsized":"2024-09-14T07:12:18.372658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:33:34.986589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"223'506","reported_seq":20881,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372972+0000","last_change":"2024-09-14T07:11:55.860412+0000","last_active":"2024-09-14T07:12:18.372972+0000","last_peered":"2024-09-14T07:12:18.372972+0000","last_clean":"2024-09-14T07:12:18.372972+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:18.372972+0000","last_undegraded":"2024-09-14T07:12:18.372972+0000","last_fullsized":"2024-09-14T07:12:18.372972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:56.003558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"222'576","reported_seq":8737,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369079+0000","last_change":"2024-09-14T07:11:55.906396+0000","last_active":"2024-09-14T07:12:18.369079+0000","last_peered":"2024-09-14T07:12:18.369079+0000","last_clean":"2024-09-14T07:12:18.369079+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:18.369079+0000","last_undegraded":"2024-09-14T07:12:18.369079+0000","last_fullsized":"2024-09-14T07:12:18.369079+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:24.818152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"222'413","reported_seq":30507,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641648+0000","last_change":"2024-09-14T07:11:55.864628+0000","last_active":"2024-09-14T07:12:18.641648+0000","last_peered":"2024-09-14T07:12:18.641648+0000","last_clean":"2024-09-14T07:12:18.641648+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.641648+0000","last_undegraded":"2024-09-14T07:12:18.641648+0000","last_fullsized":"2024-09-14T07:12:18.641648+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:48.566134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"223'1353","reported_seq":10673,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639203+0000","last_change":"2024-09-14T07:11:56.005813+0000","last_active":"2024-09-14T07:12:18.639203+0000","last_peered":"2024-09-14T07:12:18.639203+0000","last_clean":"2024-09-14T07:12:18.639203+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:18.639203+0000","last_undegraded":"2024-09-14T07:12:18.639203+0000","last_fullsized":"2024-09-14T07:12:18.639203+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:36.261505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"223'441","reported_seq":5487,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368465+0000","last_change":"2024-09-14T07:12:10.304390+0000","last_active":"2024-09-14T07:12:18.368465+0000","last_peered":"2024-09-14T07:12:18.368465+0000","last_clean":"2024-09-14T07:12:18.368465+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:18.368465+0000","last_undegraded":"2024-09-14T07:12:18.368465+0000","last_fullsized":"2024-09-14T07:12:18.368465+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:44.274673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"217'242","reported_seq":3844,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.376942+0000","last_change":"2024-09-14T07:12:18.376850+0000","last_active":"2024-09-14T07:12:18.376942+0000","last_peered":"2024-09-14T07:12:18.376942+0000","last_clean":"2024-09-14T07:12:18.376942+0000","last_became_active":"2024-09-14T07:12:18.374908+0000","last_became_peered":"2024-09-14T07:12:18.374908+0000","last_unstale":"2024-09-14T07:12:18.376942+0000","last_undegraded":"2024-09-14T07:12:18.376942+0000","last_fullsized":"2024-09-14T07:12:18.376942+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:14.574569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"218'643","reported_seq":10084,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372949+0000","last_change":"2024-09-14T07:11:55.861993+0000","last_active":"2024-09-14T07:12:18.372949+0000","last_peered":"2024-09-14T07:12:18.372949+0000","last_clean":"2024-09-14T07:12:18.372949+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:18.372949+0000","last_undegraded":"2024-09-14T07:12:18.372949+0000","last_fullsized":"2024-09-14T07:12:18.372949+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:12:03.064428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"223'538","reported_seq":7144,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368999+0000","last_change":"2024-09-14T07:11:55.928830+0000","last_active":"2024-09-14T07:12:18.368999+0000","last_peered":"2024-09-14T07:12:18.368999+0000","last_clean":"2024-09-14T07:12:18.368999+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:18.368999+0000","last_undegraded":"2024-09-14T07:12:18.368999+0000","last_fullsized":"2024-09-14T07:12:18.368999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:03:37.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"222'756","reported_seq":11149,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368752+0000","last_change":"2024-09-14T07:11:55.928041+0000","last_active":"2024-09-14T07:12:18.368752+0000","last_peered":"2024-09-14T07:12:18.368752+0000","last_clean":"2024-09-14T07:12:18.368752+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:18.368752+0000","last_undegraded":"2024-09-14T07:12:18.368752+0000","last_fullsized":"2024-09-14T07:12:18.368752+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:29:56.087478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369501+0000","last_change":"2024-09-14T07:08:31.104696+0000","last_active":"2024-09-14T07:12:18.369501+0000","last_peered":"2024-09-14T07:12:18.369501+0000","last_clean":"2024-09-14T07:12:18.369501+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:18.369501+0000","last_undegraded":"2024-09-14T07:12:18.369501+0000","last_fullsized":"2024-09-14T07:12:18.369501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:07:06.264171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'711","reported_seq":12875,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368991+0000","last_change":"2024-09-14T07:11:55.907240+0000","last_active":"2024-09-14T07:12:18.368991+0000","last_peered":"2024-09-14T07:12:18.368991+0000","last_clean":"2024-09-14T07:12:18.368991+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:18.368991+0000","last_undegraded":"2024-09-14T07:12:18.368991+0000","last_fullsized":"2024-09-14T07:12:18.368991+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:08:20.512609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639779+0000","last_change":"2024-09-14T07:10:52.791014+0000","last_active":"2024-09-14T07:12:18.639779+0000","last_peered":"2024-09-14T07:12:18.639779+0000","last_clean":"2024-09-14T07:12:18.639779+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:18.639779+0000","last_undegraded":"2024-09-14T07:12:18.639779+0000","last_fullsized":"2024-09-14T07:12:18.639779+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:53.878135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'659","reported_seq":8377,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368716+0000","last_change":"2024-09-14T07:12:10.306989+0000","last_active":"2024-09-14T07:12:18.368716+0000","last_peered":"2024-09-14T07:12:18.368716+0000","last_clean":"2024-09-14T07:12:18.368716+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:18.368716+0000","last_undegraded":"2024-09-14T07:12:18.368716+0000","last_fullsized":"2024-09-14T07:12:18.368716+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:45.664879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369005+0000","last_change":"2024-09-14T07:11:42.791395+0000","last_active":"2024-09-14T07:12:18.369005+0000","last_peered":"2024-09-14T07:12:18.369005+0000","last_clean":"2024-09-14T07:12:18.369005+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:18.369005+0000","last_undegraded":"2024-09-14T07:12:18.369005+0000","last_fullsized":"2024-09-14T07:12:18.369005+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:08.429602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"221'687","reported_seq":6739,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639507+0000","last_change":"2024-09-14T07:11:56.008517+0000","last_active":"2024-09-14T07:12:18.639507+0000","last_peered":"2024-09-14T07:12:18.639507+0000","last_clean":"2024-09-14T07:12:18.639507+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:18.639507+0000","last_undegraded":"2024-09-14T07:12:18.639507+0000","last_fullsized":"2024-09-14T07:12:18.639507+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:25:53.799687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639605+0000","last_change":"2024-09-14T07:11:42.791682+0000","last_active":"2024-09-14T07:12:18.639605+0000","last_peered":"2024-09-14T07:12:18.639605+0000","last_clean":"2024-09-14T07:12:18.639605+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:18.639605+0000","last_undegraded":"2024-09-14T07:12:18.639605+0000","last_fullsized":"2024-09-14T07:12:18.639605+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:07.695764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'756","reported_seq":4101,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372761+0000","last_change":"2024-09-14T07:11:55.861348+0000","last_active":"2024-09-14T07:12:18.372761+0000","last_peered":"2024-09-14T07:12:18.372761+0000","last_clean":"2024-09-14T07:12:18.372761+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:18.372761+0000","last_undegraded":"2024-09-14T07:12:18.372761+0000","last_fullsized":"2024-09-14T07:12:18.372761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:22:26.854178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"223'514","reported_seq":10160,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641679+0000","last_change":"2024-09-14T07:11:55.865252+0000","last_active":"2024-09-14T07:12:18.641679+0000","last_peered":"2024-09-14T07:12:18.641679+0000","last_clean":"2024-09-14T07:12:18.641679+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:18.641679+0000","last_undegraded":"2024-09-14T07:12:18.641679+0000","last_fullsized":"2024-09-14T07:12:18.641679+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:55.778064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":483,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372478+0000","last_change":"2024-09-14T07:10:52.795615+0000","last_active":"2024-09-14T07:12:18.372478+0000","last_peered":"2024-09-14T07:12:18.372478+0000","last_clean":"2024-09-14T07:12:18.372478+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:18.372478+0000","last_undegraded":"2024-09-14T07:12:18.372478+0000","last_fullsized":"2024-09-14T07:12:18.372478+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:28.723323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"221'501","reported_seq":6419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.640028+0000","last_change":"2024-09-14T07:11:56.010138+0000","last_active":"2024-09-14T07:12:18.640028+0000","last_peered":"2024-09-14T07:12:18.640028+0000","last_clean":"2024-09-14T07:12:18.640028+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:18.640028+0000","last_undegraded":"2024-09-14T07:12:18.640028+0000","last_fullsized":"2024-09-14T07:12:18.640028+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:33:52.169955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":290,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368448+0000","last_change":"2024-09-14T07:12:10.306976+0000","last_active":"2024-09-14T07:12:18.368448+0000","last_peered":"2024-09-14T07:12:18.368448+0000","last_clean":"2024-09-14T07:12:18.368448+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:18.368448+0000","last_undegraded":"2024-09-14T07:12:18.368448+0000","last_fullsized":"2024-09-14T07:12:18.368448+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:56.266310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"217'485","reported_seq":13050,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368577+0000","last_change":"2024-09-14T07:11:55.906942+0000","last_active":"2024-09-14T07:12:18.368577+0000","last_peered":"2024-09-14T07:12:18.368577+0000","last_clean":"2024-09-14T07:12:18.368577+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:18.368577+0000","last_undegraded":"2024-09-14T07:12:18.368577+0000","last_fullsized":"2024-09-14T07:12:18.368577+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:43.968427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":473,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368438+0000","last_change":"2024-09-14T07:10:52.788415+0000","last_active":"2024-09-14T07:12:18.368438+0000","last_peered":"2024-09-14T07:12:18.368438+0000","last_clean":"2024-09-14T07:12:18.368438+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:18.368438+0000","last_undegraded":"2024-09-14T07:12:18.368438+0000","last_fullsized":"2024-09-14T07:12:18.368438+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:08.172892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"222'402","reported_seq":7918,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639383+0000","last_change":"2024-09-14T07:11:56.008586+0000","last_active":"2024-09-14T07:12:18.639383+0000","last_peered":"2024-09-14T07:12:18.639383+0000","last_clean":"2024-09-14T07:12:18.639383+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:18.639383+0000","last_undegraded":"2024-09-14T07:12:18.639383+0000","last_fullsized":"2024-09-14T07:12:18.639383+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:44.933970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":482,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372692+0000","last_change":"2024-09-14T07:07:53.846433+0000","last_active":"2024-09-14T07:12:18.372692+0000","last_peered":"2024-09-14T07:12:18.372692+0000","last_clean":"2024-09-14T07:12:18.372692+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:18.372692+0000","last_undegraded":"2024-09-14T07:12:18.372692+0000","last_fullsized":"2024-09-14T07:12:18.372692+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:34.991295+0000","last_clean_scrub_stamp":"2024-09-14T07:07:34.991295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:35:03.523783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"14'72","reported_seq":585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368746+0000","last_change":"2024-09-14T07:11:42.790485+0000","last_active":"2024-09-14T07:12:18.368746+0000","last_peered":"2024-09-14T07:12:18.368746+0000","last_clean":"2024-09-14T07:12:18.368746+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:18.368746+0000","last_undegraded":"2024-09-14T07:12:18.368746+0000","last_fullsized":"2024-09-14T07:12:18.368746+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:31.282745+0000","last_clean_scrub_stamp":"2024-09-14T07:07:31.282745+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:18:05.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"223'563","reported_seq":10308,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368354+0000","last_change":"2024-09-14T07:12:10.307778+0000","last_active":"2024-09-14T07:12:18.368354+0000","last_peered":"2024-09-14T07:12:18.368354+0000","last_clean":"2024-09-14T07:12:18.368354+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:18.368354+0000","last_undegraded":"2024-09-14T07:12:18.368354+0000","last_fullsized":"2024-09-14T07:12:18.368354+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:17.108505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"223'757","reported_seq":8434,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.641712+0000","last_change":"2024-09-14T07:11:55.864626+0000","last_active":"2024-09-14T07:12:18.641712+0000","last_peered":"2024-09-14T07:12:18.641712+0000","last_clean":"2024-09-14T07:12:18.641712+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:18.641712+0000","last_undegraded":"2024-09-14T07:12:18.641712+0000","last_fullsized":"2024-09-14T07:12:18.641712+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:53.791270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"223'440","reported_seq":8268,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368658+0000","last_change":"2024-09-14T07:11:55.929683+0000","last_active":"2024-09-14T07:12:18.368658+0000","last_peered":"2024-09-14T07:12:18.368658+0000","last_clean":"2024-09-14T07:12:18.368658+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:18.368658+0000","last_undegraded":"2024-09-14T07:12:18.368658+0000","last_fullsized":"2024-09-14T07:12:18.368658+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:33.734680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"221'1024","reported_seq":8181,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.639165+0000","last_change":"2024-09-14T07:11:56.005553+0000","last_active":"2024-09-14T07:12:18.639165+0000","last_peered":"2024-09-14T07:12:18.639165+0000","last_clean":"2024-09-14T07:12:18.639165+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:18.639165+0000","last_undegraded":"2024-09-14T07:12:18.639165+0000","last_fullsized":"2024-09-14T07:12:18.639165+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:33:39.853035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"220'550","reported_seq":15528,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372744+0000","last_change":"2024-09-14T07:11:55.861555+0000","last_active":"2024-09-14T07:12:18.372744+0000","last_peered":"2024-09-14T07:12:18.372744+0000","last_clean":"2024-09-14T07:12:18.372744+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:18.372744+0000","last_undegraded":"2024-09-14T07:12:18.372744+0000","last_fullsized":"2024-09-14T07:12:18.372744+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:14.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"222'344","reported_seq":4196,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368844+0000","last_change":"2024-09-14T07:11:55.930210+0000","last_active":"2024-09-14T07:12:18.368844+0000","last_peered":"2024-09-14T07:12:18.368844+0000","last_clean":"2024-09-14T07:12:18.368844+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:18.368844+0000","last_undegraded":"2024-09-14T07:12:18.368844+0000","last_fullsized":"2024-09-14T07:12:18.368844+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T19:01:06.358294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"220'446","reported_seq":17500,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373074+0000","last_change":"2024-09-14T07:11:55.862795+0000","last_active":"2024-09-14T07:12:18.373074+0000","last_peered":"2024-09-14T07:12:18.373074+0000","last_clean":"2024-09-14T07:12:18.373074+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:18.373074+0000","last_undegraded":"2024-09-14T07:12:18.373074+0000","last_fullsized":"2024-09-14T07:12:18.373074+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:15.459268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"221'485","reported_seq":8588,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.373393+0000","last_change":"2024-09-14T07:11:55.863022+0000","last_active":"2024-09-14T07:12:18.373393+0000","last_peered":"2024-09-14T07:12:18.373393+0000","last_clean":"2024-09-14T07:12:18.373393+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:18.373393+0000","last_undegraded":"2024-09-14T07:12:18.373393+0000","last_fullsized":"2024-09-14T07:12:18.373393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:34:47.155013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"222'463","reported_seq":8201,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.372420+0000","last_change":"2024-09-14T07:11:55.860495+0000","last_active":"2024-09-14T07:12:18.372420+0000","last_peered":"2024-09-14T07:12:18.372420+0000","last_clean":"2024-09-14T07:12:18.372420+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:18.372420+0000","last_undegraded":"2024-09-14T07:12:18.372420+0000","last_fullsized":"2024-09-14T07:12:18.372420+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:01:20.398320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"223'561","reported_seq":9585,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.368461+0000","last_change":"2024-09-14T07:11:55.929529+0000","last_active":"2024-09-14T07:12:18.368461+0000","last_peered":"2024-09-14T07:12:18.368461+0000","last_clean":"2024-09-14T07:12:18.368461+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:18.368461+0000","last_undegraded":"2024-09-14T07:12:18.368461+0000","last_fullsized":"2024-09-14T07:12:18.368461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:01:02.388323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"223'562","reported_seq":7337,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.374205+0000","last_change":"2024-09-14T07:12:18.374075+0000","last_active":"2024-09-14T07:12:18.374205+0000","last_peered":"2024-09-14T07:12:18.374205+0000","last_clean":"2024-09-14T07:12:18.374205+0000","last_became_active":"2024-09-14T07:12:18.371429+0000","last_became_peered":"2024-09-14T07:12:18.371429+0000","last_unstale":"2024-09-14T07:12:18.374205+0000","last_undegraded":"2024-09-14T07:12:18.374205+0000","last_fullsized":"2024-09-14T07:12:18.374205+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:24.601874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"222'383","reported_seq":5017,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:18.369266+0000","last_change":"2024-09-14T07:11:55.928783+0000","last_active":"2024-09-14T07:12:18.369266+0000","last_peered":"2024-09-14T07:12:18.369266+0000","last_clean":"2024-09-14T07:12:18.369266+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:18.369266+0000","last_undegraded":"2024-09-14T07:12:18.369266+0000","last_fullsized":"2024-09-14T07:12:18.369266+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:07:46.759341+0000","last_clean_scrub_stamp":"2024-09-14T07:07:46.759341+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:54.519505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19507,"ondisk_log_size":19507,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82592,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289248,"statfs":{"total":96636764160,"available":96552189952,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607616,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52068,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319772,"statfs":{"total":96636764160,"available":96583446528,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607616,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70156,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301684,"statfs":{"total":96636764160,"available":96564924416,"internally_reserved":0,"allocated":2940928,"data_stored":1787422,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607617,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":2883584,"data_stored":1197054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607617,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164875,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":2860,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":2928640,"data_stored":1787441,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:30.774 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.774 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.774 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.775 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.776 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T07:07:34.991295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=34, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.777 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-14T07:07:31.282745+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=31, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.778 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.778 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.778 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.778 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.778 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.778 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.779 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.779 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.779 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.779 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.779 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.779 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T07:07:46.759341+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=7, tm_sec=46, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=12, tm_sec=26, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:12:30.779 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T07:12:50.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:12:51.107 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-14T07:12:51.108 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-14T07:12:51.124 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-14T07:12:49.288006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":257,"num_object_clones":0,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":249,"num_read":341837,"num_read_kb":1116664,"num_write":22194,"num_write_kb":331551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4057,"num_bytes_recovered":276143806,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19839,"ondisk_log_size":19839,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":340816,"kb_used_data":17008,"kb_used_omap":35,"kb_used_meta":323676,"kb_avail":565890224,"statfs":{"total":579820584960,"available":579471589376,"internally_reserved":0,"allocated":17416192,"data_stored":8363136,"data_compressed":12950,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":36329,"internal_metadata":331444759},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003678"},"pg_stats":[{"pgid":"3.1f","version":"239'394","reported_seq":7039,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:31.276539+0000","last_change":"2024-09-14T07:12:31.276539+0000","last_active":"2024-09-14T07:12:31.276539+0000","last_peered":"2024-09-14T07:12:31.276539+0000","last_clean":"2024-09-14T07:12:31.276539+0000","last_became_active":"2024-09-14T07:11:42.788423+0000","last_became_peered":"2024-09-14T07:11:42.788423+0000","last_unstale":"2024-09-14T07:12:31.276539+0000","last_undegraded":"2024-09-14T07:12:31.276539+0000","last_fullsized":"2024-09-14T07:12:31.276539+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'394","last_scrub_stamp":"2024-09-14T07:12:31.276345+0000","last_deep_scrub":"239'394","last_deep_scrub_stamp":"2024-09-14T07:12:31.276345+0000","last_clean_scrub_stamp":"2024-09-14T07:12:31.276345+0000","objects_scrubbed":12,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:18:48.001065+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0016779830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6404,"num_read_kb":50528,"num_write":556,"num_write_kb":11381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":19822624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"239'743","reported_seq":13070,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:33.747728+0000","last_change":"2024-09-14T07:12:33.747728+0000","last_active":"2024-09-14T07:12:33.747728+0000","last_peered":"2024-09-14T07:12:33.747728+0000","last_clean":"2024-09-14T07:12:33.747728+0000","last_became_active":"2024-09-14T07:12:10.301671+0000","last_became_peered":"2024-09-14T07:12:10.301671+0000","last_unstale":"2024-09-14T07:12:33.747728+0000","last_undegraded":"2024-09-14T07:12:33.747728+0000","last_fullsized":"2024-09-14T07:12:33.747728+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'743","last_scrub_stamp":"2024-09-14T07:12:33.747539+0000","last_deep_scrub":"239'743","last_deep_scrub_stamp":"2024-09-14T07:12:33.747539+0000","last_clean_scrub_stamp":"2024-09-14T07:12:33.747539+0000","objects_scrubbed":8,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:10:45.891110+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0027775230000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14206,"num_read_kb":59717,"num_write":960,"num_write_kb":21294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"239'1639","reported_seq":16764,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:39.960313+0000","last_change":"2024-09-14T07:12:39.960313+0000","last_active":"2024-09-14T07:12:39.960313+0000","last_peered":"2024-09-14T07:12:39.960313+0000","last_clean":"2024-09-14T07:12:39.960313+0000","last_became_active":"2024-09-14T07:08:33.767138+0000","last_became_peered":"2024-09-14T07:08:33.767138+0000","last_unstale":"2024-09-14T07:12:39.960313+0000","last_undegraded":"2024-09-14T07:12:39.960313+0000","last_fullsized":"2024-09-14T07:12:39.960313+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'1639","last_scrub_stamp":"2024-09-14T07:12:39.960044+0000","last_deep_scrub":"239'1639","last_deep_scrub_stamp":"2024-09-14T07:12:39.960044+0000","last_clean_scrub_stamp":"2024-09-14T07:12:39.960044+0000","objects_scrubbed":10,"log_size":1639,"log_dups_size":0,"ondisk_log_size":1639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:00:55.671123+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.002559511,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":21051,"num_read_kb":36446,"num_write":1716,"num_write_kb":5682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":136192,"num_keys_recovered":40,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"239'513","reported_seq":20896,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:38.982445+0000","last_change":"2024-09-14T07:12:38.982445+0000","last_active":"2024-09-14T07:12:38.982445+0000","last_peered":"2024-09-14T07:12:38.982445+0000","last_clean":"2024-09-14T07:12:38.982445+0000","last_became_active":"2024-09-14T07:08:31.111464+0000","last_became_peered":"2024-09-14T07:08:31.111464+0000","last_unstale":"2024-09-14T07:12:38.982445+0000","last_undegraded":"2024-09-14T07:12:38.982445+0000","last_fullsized":"2024-09-14T07:12:38.982445+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'513","last_scrub_stamp":"2024-09-14T07:12:38.982200+0000","last_deep_scrub":"239'513","last_deep_scrub_stamp":"2024-09-14T07:12:38.982200+0000","last_clean_scrub_stamp":"2024-09-14T07:12:38.982200+0000","objects_scrubbed":7,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:02:23.014717+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.001017567,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":21057,"num_read_kb":46657,"num_write":511,"num_write_kb":7501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1780736,"num_keys_recovered":42,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"239'586","reported_seq":8755,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:33.286718+0000","last_change":"2024-09-14T07:12:33.286718+0000","last_active":"2024-09-14T07:12:33.286718+0000","last_peered":"2024-09-14T07:12:33.286718+0000","last_clean":"2024-09-14T07:12:33.286718+0000","last_became_active":"2024-09-14T07:11:42.796957+0000","last_became_peered":"2024-09-14T07:11:42.796957+0000","last_unstale":"2024-09-14T07:12:33.286718+0000","last_undegraded":"2024-09-14T07:12:33.286718+0000","last_fullsized":"2024-09-14T07:12:33.286718+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'586","last_scrub_stamp":"2024-09-14T07:12:33.286525+0000","last_deep_scrub":"239'586","last_deep_scrub_stamp":"2024-09-14T07:12:33.286525+0000","last_clean_scrub_stamp":"2024-09-14T07:12:33.286525+0000","objects_scrubbed":10,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:59:05.160965+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0010385080000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8777,"num_read_kb":22848,"num_write":717,"num_write_kb":11079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":6241399,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"239'420","reported_seq":30522,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:32.174314+0000","last_change":"2024-09-14T07:12:32.174314+0000","last_active":"2024-09-14T07:12:32.174314+0000","last_peered":"2024-09-14T07:12:32.174314+0000","last_clean":"2024-09-14T07:12:32.174314+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:32.174314+0000","last_undegraded":"2024-09-14T07:12:32.174314+0000","last_fullsized":"2024-09-14T07:12:32.174314+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'420","last_scrub_stamp":"2024-09-14T07:12:32.174130+0000","last_deep_scrub":"239'420","last_deep_scrub_stamp":"2024-09-14T07:12:32.174130+0000","last_clean_scrub_stamp":"2024-09-14T07:12:32.174130+0000","objects_scrubbed":7,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:21:06.761504+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00094932600000000001,"stat_sum":{"num_bytes":19,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":29888,"num_read_kb":50743,"num_write":432,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":47832621,"num_keys_recovered":55,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"239'1359","reported_seq":10687,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:32.751852+0000","last_change":"2024-09-14T07:12:32.751852+0000","last_active":"2024-09-14T07:12:32.751852+0000","last_peered":"2024-09-14T07:12:32.751852+0000","last_clean":"2024-09-14T07:12:32.751852+0000","last_became_active":"2024-09-14T07:10:57.781949+0000","last_became_peered":"2024-09-14T07:10:57.781949+0000","last_unstale":"2024-09-14T07:12:32.751852+0000","last_undegraded":"2024-09-14T07:12:32.751852+0000","last_fullsized":"2024-09-14T07:12:32.751852+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'1359","last_scrub_stamp":"2024-09-14T07:12:32.751661+0000","last_deep_scrub":"239'1359","last_deep_scrub_stamp":"2024-09-14T07:12:32.751661+0000","last_clean_scrub_stamp":"2024-09-14T07:12:32.751661+0000","objects_scrubbed":6,"log_size":1359,"log_dups_size":0,"ondisk_log_size":1359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:51:52.342526+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0041218749999999997,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":14592,"num_read_kb":35957,"num_write":2132,"num_write_kb":9612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":7798387,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"239'449","reported_seq":5504,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:30.680521+0000","last_change":"2024-09-14T07:12:30.680433+0000","last_active":"2024-09-14T07:12:30.680521+0000","last_peered":"2024-09-14T07:12:30.680521+0000","last_clean":"2024-09-14T07:12:30.680521+0000","last_became_active":"2024-09-14T07:12:10.302439+0000","last_became_peered":"2024-09-14T07:12:10.302439+0000","last_unstale":"2024-09-14T07:12:30.680521+0000","last_undegraded":"2024-09-14T07:12:30.680521+0000","last_fullsized":"2024-09-14T07:12:30.680521+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'449","last_scrub_stamp":"2024-09-14T07:12:30.680328+0000","last_deep_scrub":"239'449","last_deep_scrub_stamp":"2024-09-14T07:12:30.680328+0000","last_clean_scrub_stamp":"2024-09-14T07:12:30.680328+0000","objects_scrubbed":8,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:33.099168+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00073739599999999997,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4886,"num_read_kb":15277,"num_write":499,"num_write_kb":15537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":2293344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"239'249","reported_seq":3859,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:30.307092+0000","last_change":"2024-09-14T07:12:30.307092+0000","last_active":"2024-09-14T07:12:30.307092+0000","last_peered":"2024-09-14T07:12:30.307092+0000","last_clean":"2024-09-14T07:12:30.307092+0000","last_became_active":"2024-09-14T07:12:18.374908+0000","last_became_peered":"2024-09-14T07:12:18.374908+0000","last_unstale":"2024-09-14T07:12:30.307092+0000","last_undegraded":"2024-09-14T07:12:30.307092+0000","last_fullsized":"2024-09-14T07:12:30.307092+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'249","last_scrub_stamp":"2024-09-14T07:12:30.307008+0000","last_deep_scrub":"239'249","last_deep_scrub_stamp":"2024-09-14T07:12:30.307008+0000","last_clean_scrub_stamp":"2024-09-14T07:12:30.307008+0000","objects_scrubbed":7,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:38:04.011645+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0069820220000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2827,"num_read_kb":20414,"num_write":244,"num_write_kb":4142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1066572,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"239'655","reported_seq":10104,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:37.067103+0000","last_change":"2024-09-14T07:12:37.067103+0000","last_active":"2024-09-14T07:12:37.067103+0000","last_peered":"2024-09-14T07:12:37.067103+0000","last_clean":"2024-09-14T07:12:37.067103+0000","last_became_active":"2024-09-14T07:11:50.829334+0000","last_became_peered":"2024-09-14T07:11:50.829334+0000","last_unstale":"2024-09-14T07:12:37.067103+0000","last_undegraded":"2024-09-14T07:12:37.067103+0000","last_fullsized":"2024-09-14T07:12:37.067103+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'655","last_scrub_stamp":"2024-09-14T07:12:37.066869+0000","last_deep_scrub":"239'655","last_deep_scrub_stamp":"2024-09-14T07:12:37.066869+0000","last_clean_scrub_stamp":"2024-09-14T07:12:37.066869+0000","objects_scrubbed":12,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:22:45.055786+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0021556100000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":10299,"num_read_kb":43418,"num_write":741,"num_write_kb":6117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":4699544,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"239'545","reported_seq":7160,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:36.863643+0000","last_change":"2024-09-14T07:12:36.863495+0000","last_active":"2024-09-14T07:12:36.863643+0000","last_peered":"2024-09-14T07:12:36.863643+0000","last_clean":"2024-09-14T07:12:36.863643+0000","last_became_active":"2024-09-14T07:11:42.791308+0000","last_became_peered":"2024-09-14T07:11:42.791308+0000","last_unstale":"2024-09-14T07:12:36.863643+0000","last_undegraded":"2024-09-14T07:12:36.863643+0000","last_fullsized":"2024-09-14T07:12:36.863643+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'545","last_scrub_stamp":"2024-09-14T07:12:36.863290+0000","last_deep_scrub":"239'545","last_deep_scrub_stamp":"2024-09-14T07:12:36.863290+0000","last_clean_scrub_stamp":"2024-09-14T07:12:36.863290+0000","objects_scrubbed":7,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:28:13.416680+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0022456939999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":7571,"num_read_kb":47364,"num_write":543,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":39474884,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"239'767","reported_seq":11169,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:35.824420+0000","last_change":"2024-09-14T07:12:35.824420+0000","last_active":"2024-09-14T07:12:35.824420+0000","last_peered":"2024-09-14T07:12:35.824420+0000","last_clean":"2024-09-14T07:12:35.824420+0000","last_became_active":"2024-09-14T07:08:31.109394+0000","last_became_peered":"2024-09-14T07:08:31.109394+0000","last_unstale":"2024-09-14T07:12:35.824420+0000","last_undegraded":"2024-09-14T07:12:35.824420+0000","last_fullsized":"2024-09-14T07:12:35.824420+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'767","last_scrub_stamp":"2024-09-14T07:12:35.824180+0000","last_deep_scrub":"239'767","last_deep_scrub_stamp":"2024-09-14T07:12:35.824180+0000","last_clean_scrub_stamp":"2024-09-14T07:12:35.824180+0000","objects_scrubbed":11,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:26:45.302004+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0016080809999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12254,"num_read_kb":39191,"num_write":926,"num_write_kb":14356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":404512,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":481,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:32.676896+0000","last_change":"2024-09-14T07:12:32.676896+0000","last_active":"2024-09-14T07:12:32.676896+0000","last_peered":"2024-09-14T07:12:32.676896+0000","last_clean":"2024-09-14T07:12:32.676896+0000","last_became_active":"2024-09-14T07:08:31.103942+0000","last_became_peered":"2024-09-14T07:08:31.103942+0000","last_unstale":"2024-09-14T07:12:32.676896+0000","last_undegraded":"2024-09-14T07:12:32.676896+0000","last_fullsized":"2024-09-14T07:12:32.676896+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:12:32.676707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:12:32.676707+0000","last_clean_scrub_stamp":"2024-09-14T07:12:32.676707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:32.065436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072199,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"239'715","reported_seq":12887,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:32.263133+0000","last_change":"2024-09-14T07:12:32.263133+0000","last_active":"2024-09-14T07:12:32.263133+0000","last_peered":"2024-09-14T07:12:32.263133+0000","last_clean":"2024-09-14T07:12:32.263133+0000","last_became_active":"2024-09-14T07:11:45.799067+0000","last_became_peered":"2024-09-14T07:11:45.799067+0000","last_unstale":"2024-09-14T07:12:32.263133+0000","last_undegraded":"2024-09-14T07:12:32.263133+0000","last_fullsized":"2024-09-14T07:12:32.263133+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'715","last_scrub_stamp":"2024-09-14T07:12:32.262895+0000","last_deep_scrub":"239'715","last_deep_scrub_stamp":"2024-09-14T07:12:32.262895+0000","last_clean_scrub_stamp":"2024-09-14T07:12:32.262895+0000","objects_scrubbed":4,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:13.677725+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0012314439999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":14043,"num_read_kb":35165,"num_write":752,"num_write_kb":5118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":6612790,"num_keys_recovered":77,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":175,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:28.713557+0000","last_change":"2024-09-14T07:12:28.713557+0000","last_active":"2024-09-14T07:12:28.713557+0000","last_peered":"2024-09-14T07:12:28.713557+0000","last_clean":"2024-09-14T07:12:28.713557+0000","last_became_active":"2024-09-14T07:10:52.790529+0000","last_became_peered":"2024-09-14T07:10:52.790529+0000","last_unstale":"2024-09-14T07:12:28.713557+0000","last_undegraded":"2024-09-14T07:12:28.713557+0000","last_fullsized":"2024-09-14T07:12:28.713557+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:12:28.713484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:12:28.713484+0000","last_clean_scrub_stamp":"2024-09-14T07:12:28.713484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:14:11.600886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"239'666","reported_seq":8392,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:29.692226+0000","last_change":"2024-09-14T07:12:29.692226+0000","last_active":"2024-09-14T07:12:29.692226+0000","last_peered":"2024-09-14T07:12:29.692226+0000","last_clean":"2024-09-14T07:12:29.692226+0000","last_became_active":"2024-09-14T07:12:10.305572+0000","last_became_peered":"2024-09-14T07:12:10.305572+0000","last_unstale":"2024-09-14T07:12:29.692226+0000","last_undegraded":"2024-09-14T07:12:29.692226+0000","last_fullsized":"2024-09-14T07:12:29.692226+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'666","last_scrub_stamp":"2024-09-14T07:12:29.692091+0000","last_deep_scrub":"239'666","last_deep_scrub_stamp":"2024-09-14T07:12:29.692091+0000","last_clean_scrub_stamp":"2024-09-14T07:12:29.692091+0000","objects_scrubbed":7,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:14:01.167914+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0048386999999999996,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9674,"num_read_kb":41023,"num_write":702,"num_write_kb":6324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":3560672,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":490,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:31.658057+0000","last_change":"2024-09-14T07:12:31.658057+0000","last_active":"2024-09-14T07:12:31.658057+0000","last_peered":"2024-09-14T07:12:31.658057+0000","last_clean":"2024-09-14T07:12:31.658057+0000","last_became_active":"2024-09-14T07:11:42.791202+0000","last_became_peered":"2024-09-14T07:11:42.791202+0000","last_unstale":"2024-09-14T07:12:31.658057+0000","last_undegraded":"2024-09-14T07:12:31.658057+0000","last_fullsized":"2024-09-14T07:12:31.658057+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:12:31.657874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:12:31.657874+0000","last_clean_scrub_stamp":"2024-09-14T07:12:31.657874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:54:29.917632+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00099469599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"239'696","reported_seq":6756,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:33.783070+0000","last_change":"2024-09-14T07:12:33.783070+0000","last_active":"2024-09-14T07:12:33.783070+0000","last_peered":"2024-09-14T07:12:33.783070+0000","last_clean":"2024-09-14T07:12:33.783070+0000","last_became_active":"2024-09-14T07:10:56.234498+0000","last_became_peered":"2024-09-14T07:10:56.234498+0000","last_unstale":"2024-09-14T07:12:33.783070+0000","last_undegraded":"2024-09-14T07:12:33.783070+0000","last_fullsized":"2024-09-14T07:12:33.783070+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'696","last_scrub_stamp":"2024-09-14T07:12:33.782801+0000","last_deep_scrub":"239'696","last_deep_scrub_stamp":"2024-09-14T07:12:33.782801+0000","last_clean_scrub_stamp":"2024-09-14T07:12:33.782801+0000","objects_scrubbed":9,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:37:26.179996+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0067295469999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7771,"num_read_kb":17350,"num_write":863,"num_write_kb":4513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":2818702,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":184,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:29.713141+0000","last_change":"2024-09-14T07:12:29.713141+0000","last_active":"2024-09-14T07:12:29.713141+0000","last_peered":"2024-09-14T07:12:29.713141+0000","last_clean":"2024-09-14T07:12:29.713141+0000","last_became_active":"2024-09-14T07:11:42.791419+0000","last_became_peered":"2024-09-14T07:11:42.791419+0000","last_unstale":"2024-09-14T07:12:29.713141+0000","last_undegraded":"2024-09-14T07:12:29.713141+0000","last_fullsized":"2024-09-14T07:12:29.713141+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:12:29.713064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:12:29.713064+0000","last_clean_scrub_stamp":"2024-09-14T07:12:29.713064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:03:08.895897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001334444,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"239'759","reported_seq":4112,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:33.063390+0000","last_change":"2024-09-14T07:12:33.063390+0000","last_active":"2024-09-14T07:12:33.063390+0000","last_peered":"2024-09-14T07:12:33.063390+0000","last_clean":"2024-09-14T07:12:33.063390+0000","last_became_active":"2024-09-14T07:07:53.836447+0000","last_became_peered":"2024-09-14T07:07:53.836447+0000","last_unstale":"2024-09-14T07:12:33.063390+0000","last_undegraded":"2024-09-14T07:12:33.063390+0000","last_fullsized":"2024-09-14T07:12:33.063390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'759","last_scrub_stamp":"2024-09-14T07:12:33.063201+0000","last_deep_scrub":"239'759","last_deep_scrub_stamp":"2024-09-14T07:12:33.063201+0000","last_clean_scrub_stamp":"2024-09-14T07:12:33.063201+0000","objects_scrubbed":3,"log_size":759,"log_dups_size":0,"ondisk_log_size":759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:00:46.300153+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0015292319999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5233,"num_read_kb":19721,"num_write":1116,"num_write_kb":5803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":139264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"239'519","reported_seq":10174,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:31.175545+0000","last_change":"2024-09-14T07:12:31.175412+0000","last_active":"2024-09-14T07:12:31.175545+0000","last_peered":"2024-09-14T07:12:31.175545+0000","last_clean":"2024-09-14T07:12:31.175545+0000","last_became_active":"2024-09-14T07:11:45.800827+0000","last_became_peered":"2024-09-14T07:11:45.800827+0000","last_unstale":"2024-09-14T07:12:31.175545+0000","last_undegraded":"2024-09-14T07:12:31.175545+0000","last_fullsized":"2024-09-14T07:12:31.175545+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'519","last_scrub_stamp":"2024-09-14T07:12:31.175218+0000","last_deep_scrub":"239'519","last_deep_scrub_stamp":"2024-09-14T07:12:31.175218+0000","last_clean_scrub_stamp":"2024-09-14T07:12:31.175218+0000","objects_scrubbed":5,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:57.681715+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0014030340000000001,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8760,"num_read_kb":48219,"num_write":553,"num_write_kb":16939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":28493824,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"239'3","reported_seq":492,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:31.131792+0000","last_change":"2024-09-14T07:12:31.131792+0000","last_active":"2024-09-14T07:12:31.131792+0000","last_peered":"2024-09-14T07:12:31.131792+0000","last_clean":"2024-09-14T07:12:31.131792+0000","last_became_active":"2024-09-14T07:10:52.790840+0000","last_became_peered":"2024-09-14T07:10:52.790840+0000","last_unstale":"2024-09-14T07:12:31.131792+0000","last_undegraded":"2024-09-14T07:12:31.131792+0000","last_fullsized":"2024-09-14T07:12:31.131792+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'3","last_scrub_stamp":"2024-09-14T07:12:31.131604+0000","last_deep_scrub":"239'3","last_deep_scrub_stamp":"2024-09-14T07:12:31.131604+0000","last_clean_scrub_stamp":"2024-09-14T07:12:31.131604+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:16:28.131187+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001993725,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"239'504","reported_seq":6430,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:30.730082+0000","last_change":"2024-09-14T07:12:30.730082+0000","last_active":"2024-09-14T07:12:30.730082+0000","last_peered":"2024-09-14T07:12:30.730082+0000","last_clean":"2024-09-14T07:12:30.730082+0000","last_became_active":"2024-09-14T07:10:59.789452+0000","last_became_peered":"2024-09-14T07:10:59.789452+0000","last_unstale":"2024-09-14T07:12:30.730082+0000","last_undegraded":"2024-09-14T07:12:30.730082+0000","last_fullsized":"2024-09-14T07:12:30.730082+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'504","last_scrub_stamp":"2024-09-14T07:12:30.729982+0000","last_deep_scrub":"239'504","last_deep_scrub_stamp":"2024-09-14T07:12:30.729982+0000","last_clean_scrub_stamp":"2024-09-14T07:12:30.729982+0000","objects_scrubbed":3,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:48:06.323181+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.007268984,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":7619,"num_read_kb":23804,"num_write":402,"num_write_kb":4399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":4931863,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":298,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:32.724437+0000","last_change":"2024-09-14T07:12:32.724437+0000","last_active":"2024-09-14T07:12:32.724437+0000","last_peered":"2024-09-14T07:12:32.724437+0000","last_clean":"2024-09-14T07:12:32.724437+0000","last_became_active":"2024-09-14T07:12:10.306171+0000","last_became_peered":"2024-09-14T07:12:10.306171+0000","last_unstale":"2024-09-14T07:12:32.724437+0000","last_undegraded":"2024-09-14T07:12:32.724437+0000","last_fullsized":"2024-09-14T07:12:32.724437+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:12:32.724221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:12:32.724221+0000","last_clean_scrub_stamp":"2024-09-14T07:12:32.724221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:32.593181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"239'494","reported_seq":13067,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:28.255384+0000","last_change":"2024-09-14T07:12:28.255384+0000","last_active":"2024-09-14T07:12:28.255384+0000","last_peered":"2024-09-14T07:12:28.255384+0000","last_clean":"2024-09-14T07:12:28.255384+0000","last_became_active":"2024-09-14T07:11:42.793720+0000","last_became_peered":"2024-09-14T07:11:42.793720+0000","last_unstale":"2024-09-14T07:12:28.255384+0000","last_undegraded":"2024-09-14T07:12:28.255384+0000","last_fullsized":"2024-09-14T07:12:28.255384+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'494","last_scrub_stamp":"2024-09-14T07:12:28.255303+0000","last_deep_scrub":"239'494","last_deep_scrub_stamp":"2024-09-14T07:12:28.255303+0000","last_clean_scrub_stamp":"2024-09-14T07:12:28.255303+0000","objects_scrubbed":9,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:06:19.092159+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0013684229999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":13003,"num_read_kb":19483,"num_write":384,"num_write_kb":7935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":270971,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":481,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:30.684910+0000","last_change":"2024-09-14T07:12:30.684910+0000","last_active":"2024-09-14T07:12:30.684910+0000","last_peered":"2024-09-14T07:12:30.684910+0000","last_clean":"2024-09-14T07:12:30.684910+0000","last_became_active":"2024-09-14T07:10:52.787575+0000","last_became_peered":"2024-09-14T07:10:52.787575+0000","last_unstale":"2024-09-14T07:12:30.684910+0000","last_undegraded":"2024-09-14T07:12:30.684910+0000","last_fullsized":"2024-09-14T07:12:30.684910+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:12:30.684727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:12:30.684727+0000","last_clean_scrub_stamp":"2024-09-14T07:12:30.684727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:45:52.364040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001581654,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"239'408","reported_seq":7932,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:31.761756+0000","last_change":"2024-09-14T07:12:31.761756+0000","last_active":"2024-09-14T07:12:31.761756+0000","last_peered":"2024-09-14T07:12:31.761756+0000","last_clean":"2024-09-14T07:12:31.761756+0000","last_became_active":"2024-09-14T07:11:50.827209+0000","last_became_peered":"2024-09-14T07:11:50.827209+0000","last_unstale":"2024-09-14T07:12:31.761756+0000","last_undegraded":"2024-09-14T07:12:31.761756+0000","last_fullsized":"2024-09-14T07:12:31.761756+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'408","last_scrub_stamp":"2024-09-14T07:12:31.761549+0000","last_deep_scrub":"239'408","last_deep_scrub_stamp":"2024-09-14T07:12:31.761549+0000","last_clean_scrub_stamp":"2024-09-14T07:12:31.761549+0000","objects_scrubbed":6,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:48:56.442902+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.006516577,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":6439,"num_read_kb":16011,"num_write":346,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":387,"num_bytes_recovered":3525271,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"239'2","reported_seq":491,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:32.105718+0000","last_change":"2024-09-14T07:12:32.105718+0000","last_active":"2024-09-14T07:12:32.105718+0000","last_peered":"2024-09-14T07:12:32.105718+0000","last_clean":"2024-09-14T07:12:32.105718+0000","last_became_active":"2024-09-14T07:07:53.836348+0000","last_became_peered":"2024-09-14T07:07:53.836348+0000","last_unstale":"2024-09-14T07:12:32.105718+0000","last_undegraded":"2024-09-14T07:12:32.105718+0000","last_fullsized":"2024-09-14T07:12:32.105718+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'2","last_scrub_stamp":"2024-09-14T07:12:32.105528+0000","last_deep_scrub":"239'2","last_deep_scrub_stamp":"2024-09-14T07:12:32.105528+0000","last_clean_scrub_stamp":"2024-09-14T07:12:32.105528+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:40:50.896599+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00091234599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"239'74","reported_seq":595,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:29.663151+0000","last_change":"2024-09-14T07:12:29.663151+0000","last_active":"2024-09-14T07:12:29.663151+0000","last_peered":"2024-09-14T07:12:29.663151+0000","last_clean":"2024-09-14T07:12:29.663151+0000","last_became_active":"2024-09-14T07:11:42.787734+0000","last_became_peered":"2024-09-14T07:11:42.787734+0000","last_unstale":"2024-09-14T07:12:29.663151+0000","last_undegraded":"2024-09-14T07:12:29.663151+0000","last_fullsized":"2024-09-14T07:12:29.663151+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'74","last_scrub_stamp":"2024-09-14T07:12:29.662963+0000","last_deep_scrub":"239'74","last_deep_scrub_stamp":"2024-09-14T07:12:29.662963+0000","last_clean_scrub_stamp":"2024-09-14T07:12:29.662963+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:18:04.239711+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"239'570","reported_seq":10323,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:31.700214+0000","last_change":"2024-09-14T07:12:31.700214+0000","last_active":"2024-09-14T07:12:31.700214+0000","last_peered":"2024-09-14T07:12:31.700214+0000","last_clean":"2024-09-14T07:12:31.700214+0000","last_became_active":"2024-09-14T07:12:10.305914+0000","last_became_peered":"2024-09-14T07:12:10.305914+0000","last_unstale":"2024-09-14T07:12:31.700214+0000","last_undegraded":"2024-09-14T07:12:31.700214+0000","last_fullsized":"2024-09-14T07:12:31.700214+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'570","last_scrub_stamp":"2024-09-14T07:12:31.699997+0000","last_deep_scrub":"239'570","last_deep_scrub_stamp":"2024-09-14T07:12:31.699997+0000","last_clean_scrub_stamp":"2024-09-14T07:12:31.699997+0000","objects_scrubbed":7,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:53.280442+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0027459770000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":9749,"num_read_kb":69215,"num_write":706,"num_write_kb":27479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":30174576,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"239'766","reported_seq":8451,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:30.177871+0000","last_change":"2024-09-14T07:12:30.177871+0000","last_active":"2024-09-14T07:12:30.177871+0000","last_peered":"2024-09-14T07:12:30.177871+0000","last_clean":"2024-09-14T07:12:30.177871+0000","last_became_active":"2024-09-14T07:08:33.766500+0000","last_became_peered":"2024-09-14T07:08:33.766500+0000","last_unstale":"2024-09-14T07:12:30.177871+0000","last_undegraded":"2024-09-14T07:12:30.177871+0000","last_fullsized":"2024-09-14T07:12:30.177871+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'766","last_scrub_stamp":"2024-09-14T07:12:30.177682+0000","last_deep_scrub":"239'766","last_deep_scrub_stamp":"2024-09-14T07:12:30.177682+0000","last_clean_scrub_stamp":"2024-09-14T07:12:30.177682+0000","objects_scrubbed":9,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:58.931245+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00089031099999999997,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10196,"num_read_kb":48135,"num_write":767,"num_write_kb":17473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":286207,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"239'447","reported_seq":8283,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:33.730328+0000","last_change":"2024-09-14T07:12:33.730328+0000","last_active":"2024-09-14T07:12:33.730328+0000","last_peered":"2024-09-14T07:12:33.730328+0000","last_clean":"2024-09-14T07:12:33.730328+0000","last_became_active":"2024-09-14T07:08:31.108848+0000","last_became_peered":"2024-09-14T07:08:31.108848+0000","last_unstale":"2024-09-14T07:12:33.730328+0000","last_undegraded":"2024-09-14T07:12:33.730328+0000","last_fullsized":"2024-09-14T07:12:33.730328+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'447","last_scrub_stamp":"2024-09-14T07:12:33.730142+0000","last_deep_scrub":"239'447","last_deep_scrub_stamp":"2024-09-14T07:12:33.730142+0000","last_clean_scrub_stamp":"2024-09-14T07:12:33.730142+0000","objects_scrubbed":7,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:55.018860+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0027605379999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":7828,"num_read_kb":35945,"num_write":416,"num_write_kb":4786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":503808,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"239'1034","reported_seq":8199,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:34.752536+0000","last_change":"2024-09-14T07:12:34.752536+0000","last_active":"2024-09-14T07:12:34.752536+0000","last_peered":"2024-09-14T07:12:34.752536+0000","last_clean":"2024-09-14T07:12:34.752536+0000","last_became_active":"2024-09-14T07:11:42.791542+0000","last_became_peered":"2024-09-14T07:11:42.791542+0000","last_unstale":"2024-09-14T07:12:34.752536+0000","last_undegraded":"2024-09-14T07:12:34.752536+0000","last_fullsized":"2024-09-14T07:12:34.752536+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'1034","last_scrub_stamp":"2024-09-14T07:12:34.752329+0000","last_deep_scrub":"239'1034","last_deep_scrub_stamp":"2024-09-14T07:12:34.752329+0000","last_clean_scrub_stamp":"2024-09-14T07:12:34.752329+0000","objects_scrubbed":10,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:09.844059+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0038739500000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":10239,"num_read_kb":41587,"num_write":1383,"num_write_kb":13969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":36418223,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"239'561","reported_seq":15547,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:34.063309+0000","last_change":"2024-09-14T07:12:34.063309+0000","last_active":"2024-09-14T07:12:34.063309+0000","last_peered":"2024-09-14T07:12:34.063309+0000","last_clean":"2024-09-14T07:12:34.063309+0000","last_became_active":"2024-09-14T07:11:42.794558+0000","last_became_peered":"2024-09-14T07:11:42.794558+0000","last_unstale":"2024-09-14T07:12:34.063309+0000","last_undegraded":"2024-09-14T07:12:34.063309+0000","last_fullsized":"2024-09-14T07:12:34.063309+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'561","last_scrub_stamp":"2024-09-14T07:12:34.063104+0000","last_deep_scrub":"239'561","last_deep_scrub_stamp":"2024-09-14T07:12:34.063104+0000","last_clean_scrub_stamp":"2024-09-14T07:12:34.063104+0000","objects_scrubbed":11,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:25:06.849915+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001817503,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":16521,"num_read_kb":38026,"num_write":464,"num_write_kb":7977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1001472,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"239'352","reported_seq":4212,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:34.779735+0000","last_change":"2024-09-14T07:12:34.779735+0000","last_active":"2024-09-14T07:12:34.779735+0000","last_peered":"2024-09-14T07:12:34.779735+0000","last_clean":"2024-09-14T07:12:34.779735+0000","last_became_active":"2024-09-14T07:07:53.833481+0000","last_became_peered":"2024-09-14T07:07:53.833481+0000","last_unstale":"2024-09-14T07:12:34.779735+0000","last_undegraded":"2024-09-14T07:12:34.779735+0000","last_fullsized":"2024-09-14T07:12:34.779735+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'352","last_scrub_stamp":"2024-09-14T07:12:34.779472+0000","last_deep_scrub":"239'352","last_deep_scrub_stamp":"2024-09-14T07:12:34.779472+0000","last_clean_scrub_stamp":"2024-09-14T07:12:34.779472+0000","objects_scrubbed":8,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:27.647533+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0031335529999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3366,"num_read_kb":19030,"num_write":325,"num_write_kb":4904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"239'454","reported_seq":17516,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:35.089937+0000","last_change":"2024-09-14T07:12:35.089937+0000","last_active":"2024-09-14T07:12:35.089937+0000","last_peered":"2024-09-14T07:12:35.089937+0000","last_clean":"2024-09-14T07:12:35.089937+0000","last_became_active":"2024-09-14T07:11:50.837316+0000","last_became_peered":"2024-09-14T07:11:50.837316+0000","last_unstale":"2024-09-14T07:12:35.089937+0000","last_undegraded":"2024-09-14T07:12:35.089937+0000","last_fullsized":"2024-09-14T07:12:35.089937+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'454","last_scrub_stamp":"2024-09-14T07:12:35.089730+0000","last_deep_scrub":"239'454","last_deep_scrub_stamp":"2024-09-14T07:12:35.089730+0000","last_clean_scrub_stamp":"2024-09-14T07:12:35.089730+0000","objects_scrubbed":8,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:57:59.492613+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0027647349999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":17549,"num_read_kb":39505,"num_write":439,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7992552,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"239'497","reported_seq":8608,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:36.065400+0000","last_change":"2024-09-14T07:12:36.065400+0000","last_active":"2024-09-14T07:12:36.065400+0000","last_peered":"2024-09-14T07:12:36.065400+0000","last_clean":"2024-09-14T07:12:36.065400+0000","last_became_active":"2024-09-14T07:07:47.790940+0000","last_became_peered":"2024-09-14T07:07:47.790940+0000","last_unstale":"2024-09-14T07:12:36.065400+0000","last_undegraded":"2024-09-14T07:12:36.065400+0000","last_fullsized":"2024-09-14T07:12:36.065400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'497","last_scrub_stamp":"2024-09-14T07:12:36.065186+0000","last_deep_scrub":"239'497","last_deep_scrub_stamp":"2024-09-14T07:12:36.065186+0000","last_clean_scrub_stamp":"2024-09-14T07:12:36.065186+0000","objects_scrubbed":12,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:28.712700+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0.0027979910000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":8449,"num_read_kb":14252,"num_write":616,"num_write_kb":16879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"239'469","reported_seq":8215,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:38.023054+0000","last_change":"2024-09-14T07:12:38.023054+0000","last_active":"2024-09-14T07:12:38.023054+0000","last_peered":"2024-09-14T07:12:38.023054+0000","last_clean":"2024-09-14T07:12:38.023054+0000","last_became_active":"2024-09-14T07:11:50.827007+0000","last_became_peered":"2024-09-14T07:11:50.827007+0000","last_unstale":"2024-09-14T07:12:38.023054+0000","last_undegraded":"2024-09-14T07:12:38.023054+0000","last_fullsized":"2024-09-14T07:12:38.023054+0000","mapping_epoch":217,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":218,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'469","last_scrub_stamp":"2024-09-14T07:12:38.022868+0000","last_deep_scrub":"239'469","last_deep_scrub_stamp":"2024-09-14T07:12:38.022868+0000","last_clean_scrub_stamp":"2024-09-14T07:12:38.022868+0000","objects_scrubbed":6,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:34:29.436472+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001148336,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7121,"num_read_kb":28848,"num_write":652,"num_write_kb":17683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":5351620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"239'570","reported_seq":9603,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:38.858228+0000","last_change":"2024-09-14T07:12:38.858228+0000","last_active":"2024-09-14T07:12:38.858228+0000","last_peered":"2024-09-14T07:12:38.858228+0000","last_clean":"2024-09-14T07:12:38.858228+0000","last_became_active":"2024-09-14T07:08:31.109488+0000","last_became_peered":"2024-09-14T07:08:31.109488+0000","last_unstale":"2024-09-14T07:12:38.858228+0000","last_undegraded":"2024-09-14T07:12:38.858228+0000","last_fullsized":"2024-09-14T07:12:38.858228+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'570","last_scrub_stamp":"2024-09-14T07:12:38.857967+0000","last_deep_scrub":"239'570","last_deep_scrub_stamp":"2024-09-14T07:12:38.857967+0000","last_clean_scrub_stamp":"2024-09-14T07:12:38.857967+0000","objects_scrubbed":9,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:20.599711+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.0027461690000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10659,"num_read_kb":24574,"num_write":438,"num_write_kb":9187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":1328414,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"239'571","reported_seq":7354,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:29.267541+0000","last_change":"2024-09-14T07:12:29.267541+0000","last_active":"2024-09-14T07:12:29.267541+0000","last_peered":"2024-09-14T07:12:29.267541+0000","last_clean":"2024-09-14T07:12:29.267541+0000","last_became_active":"2024-09-14T07:12:18.371429+0000","last_became_peered":"2024-09-14T07:12:18.371429+0000","last_unstale":"2024-09-14T07:12:29.267541+0000","last_undegraded":"2024-09-14T07:12:29.267541+0000","last_fullsized":"2024-09-14T07:12:29.267541+0000","mapping_epoch":238,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":239,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'571","last_scrub_stamp":"2024-09-14T07:12:29.267270+0000","last_deep_scrub":"239'571","last_deep_scrub_stamp":"2024-09-14T07:12:29.267270+0000","last_clean_scrub_stamp":"2024-09-14T07:12:29.267270+0000","objects_scrubbed":9,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:17:12.035501+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0016651700000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7830,"num_read_kb":22930,"num_write":432,"num_write_kb":7521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":3609600,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"239'389","reported_seq":5031,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-14T07:12:37.864791+0000","last_change":"2024-09-14T07:12:37.864791+0000","last_active":"2024-09-14T07:12:37.864791+0000","last_peered":"2024-09-14T07:12:37.864791+0000","last_clean":"2024-09-14T07:12:37.864791+0000","last_became_active":"2024-09-14T07:10:02.770825+0000","last_became_peered":"2024-09-14T07:10:02.770825+0000","last_unstale":"2024-09-14T07:12:37.864791+0000","last_undegraded":"2024-09-14T07:12:37.864791+0000","last_fullsized":"2024-09-14T07:12:37.864791+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'389","last_scrub_stamp":"2024-09-14T07:12:37.864588+0000","last_deep_scrub":"239'389","last_deep_scrub_stamp":"2024-09-14T07:12:37.864588+0000","last_clean_scrub_stamp":"2024-09-14T07:12:37.864588+0000","objects_scrubbed":6,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:15:07.508018+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0020829249999999998,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":5910,"num_read_kb":45226,"num_write":358,"num_write_kb":5493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":6392388,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":253,"num_object_clones":0,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":249,"num_read":341771,"num_read_kb":1116609,"num_write":22091,"num_write_kb":329921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4049,"num_bytes_recovered":274963032,"num_keys_recovered":677,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},"log_size":19760,"ondisk_log_size":19760,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":19842,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82580,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289260,"statfs":{"total":96636764160,"available":96552202240,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":12,"seq":51539607620,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52052,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319788,"statfs":{"total":96636764160,"available":96583462912,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607620,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74312,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297528,"statfs":{"total":96636764160,"available":96560668672,"internally_reserved":0,"allocated":2940928,"data_stored":1787422,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607621,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47888,"kb_used_data":2816,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323952,"statfs":{"total":96636764160,"available":96587726848,"internally_reserved":0,"allocated":2883584,"data_stored":1197054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607621,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47892,"kb_used_data":2820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323948,"statfs":{"total":96636764160,"available":96587722752,"internally_reserved":0,"allocated":2887680,"data_stored":1197073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":209,"seq":897648164879,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":2860,"kb_used_omap":27,"kb_used_meta":33188,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":2928640,"data_stored":1787441,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":28356,"internal_metadata":33984828},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":6892,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6058,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12989,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:12:51.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T07:12:51.467 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T07:12:51.467 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T07:12:51.467 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T07:12:51.467 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:12:51.544 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T07:12:51.544 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T07:12:51.544 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:12:52.571 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-14T07:12:57.646 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T07:12:57.647 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T07:12:57.647 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:12:57.718 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T07:12:57.719 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T07:12:57.719 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:12:57.810 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T07:12:57.810 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T07:12:57.811 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:12:57.895 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T07:12:57.896 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T07:12:57.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:12:58.378 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~6s 2024-09-14T07:13:03.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-14T07:13:03.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-14T07:13:03.998 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T07:13:03.998 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T07:13:03.998 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T07:13:03.998 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:13:04.029 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T07:13:04.029 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T07:13:04.029 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T07:13:04.029 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:13:04.048 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T07:13:04.049 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T07:13:04.049 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:13:04.115 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T07:13:04.115 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T07:13:04.115 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:13:04.140 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T07:13:04.140 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T07:13:04.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T07:13:04.177 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:13:04.178 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T07:13:04.419 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:13:04.419 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T07:13:04.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:13:04.599 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T07:13:04.741 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:13:04.741 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T07:13:05.033 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:13:05.034 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T07:13:05.296 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:13:05.296 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T07:13:05.486 INFO:tasks.ceph:Archiving mon data... 2024-09-14T07:13:05.487 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/data/mon.a.tgz 2024-09-14T07:13:05.487 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-14T07:13:05.493 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.EUdTK56wOg 2024-09-14T07:13:05.493 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.EUdTK56wOg 2024-09-14T07:13:05.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.EUdTK56wOg 2024-09-14T07:13:05.724 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.EUdTK56wOg is 437KB 2024-09-14T07:13:05.785 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.EUdTK56wOg 2024-09-14T07:13:05.790 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/data/mon.c.tgz 2024-09-14T07:13:05.790 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-14T07:13:05.836 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.Eva4j75U26 2024-09-14T07:13:05.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Eva4j75U26 2024-09-14T07:13:06.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.Eva4j75U26 2024-09-14T07:13:06.103 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.Eva4j75U26 is 583KB 2024-09-14T07:13:06.169 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.Eva4j75U26 2024-09-14T07:13:06.174 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/data/mon.b.tgz 2024-09-14T07:13:06.175 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-14T07:13:06.181 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.GQnfOIVJlo 2024-09-14T07:13:06.181 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.GQnfOIVJlo 2024-09-14T07:13:06.374 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.GQnfOIVJlo 2024-09-14T07:13:06.439 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.GQnfOIVJlo is 583KB 2024-09-14T07:13:06.502 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.GQnfOIVJlo 2024-09-14T07:13:06.507 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T07:13:06.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T07:13:06.509 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T07:13:06.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T07:13:06.573 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T07:13:06.613 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T07:13:06.618 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T07:13:06.618 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T07:13:06.619 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi005/crash 2024-09-14T07:13:06.620 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T07:13:06.634 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi110/crash 2024-09-14T07:13:06.635 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T07:13:06.668 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi145/crash 2024-09-14T07:13:06.668 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T07:13:06.680 INFO:tasks.ceph:Compressing logs... 2024-09-14T07:13:06.681 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T07:13:06.683 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T07:13:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34864.log 2024-09-14T07:13:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29478.log 2024-09-14T07:13:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34864.log.gz 2024-09-14T07:13:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40924.log 2024-09-14T07:13:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34770.log 2024-09-14T07:13:06.702 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29478.log.gz 2024-09-14T07:13:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40924.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40924.log.gz --verbose 2024-09-14T07:13:06.702 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.33460.log 2024-09-14T07:13:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28122.log 2024-09-14T07:13:06.703 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34770.log.gz 2024-09-14T07:13:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33460.log.gz 2024-09-14T07:13:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.39803.log 2024-09-14T07:13:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28122.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.28004.log -- replaced with /var/log/ceph/ceph-client.admin.28122.log.gz 2024-09-14T07:13:06.704 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-14T07:13:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/objectstore_tool.39803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37206.log 2024-09-14T07:13:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28004.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.28004.log.gz -5 2024-09-14T07:13:06.705 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35921.log 2024-09-14T07:13:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32227.log 2024-09-14T07:13:06.705 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37206.log.gz 2024-09-14T07:13:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28436.log 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35921.log.gz 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32227.log.gz 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28123.log 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35924.log 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28436.log.gz 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34005.log 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28123.log.gz 2024-09-14T07:13:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34133.log 2024-09-14T07:13:06.708 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35924.log.gz 2024-09-14T07:13:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30088.log 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34005.log.gz 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43400.log 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34133.log.gz 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41854.log 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30088.log.gz 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25155.log 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43400.log.gz 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30282.log 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41854.log.gz 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.25155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32602.log 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25155.log.gz 2024-09-14T07:13:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29279.log 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30282.log.gz 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28918.log 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32602.log.gz 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33577.log 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29279.log.gz 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37879.log 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28918.log.gz 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43246.log 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33577.log.gz 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29918.log 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37879.log.gz 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37415.log 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43246.log.gz 2024-09-14T07:13:06.715 INFO:teuthology.orchestra.run.smithi005.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.39803.log.gz 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30160.log 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29918.log.gz 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30059.log 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37415.log.gz 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30160.log.gz 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30059.log.gz 2024-09-14T07:13:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27984.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43383.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.27984.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.38978.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27984.log.gz 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43383.log.gz 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43592.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/objectstore_tool.38978.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38455.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43592.log.gz 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36634.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32301.log 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38455.log.gz 2024-09-14T07:13:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37728.log 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36634.log.gz 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32301.log.gz 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39008.log 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37728.log.gz 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39008.log.gz 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36581.log 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36774.log 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36080.log 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42499.log 2024-09-14T07:13:06.723 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36774.log.gz 2024-09-14T07:13:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34859.log 2024-09-14T07:13:06.724 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36080.log.gz 2024-09-14T07:13:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42499.log.gz 2024-09-14T07:13:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2024-09-14T07:13:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34859.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.30440.log 2024-09-14T07:13:06.724 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34859.log.gz 2024-09-14T07:13:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33939.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz -5 2024-09-14T07:13:06.725 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43244.log 2024-09-14T07:13:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40641.log 2024-09-14T07:13:06.725 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30440.log.gz 2024-09-14T07:13:06.726 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T07:13:06.742 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25140.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25116.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25140.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25140.log.gz 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25116.log.gz 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25212.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25188.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25212.log.gz 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T07:13:06.743 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.admin.25188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25188.log.gz 2024-09-14T07:13:06.744 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25236.log 2024-09-14T07:13:06.744 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T07:13:06.744 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.25236.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T07:13:06.744 INFO:teuthology.orchestra.run.smithi110.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25236.log.gz 2024-09-14T07:13:06.744 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25164.log 2024-09-14T07:13:06.747 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.25164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25164.log.gz 2024-09-14T07:13:06.747 INFO:teuthology.orchestra.run.smithi110.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T07:13:06.747 INFO:teuthology.orchestra.run.smithi110.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T07:13:06.753 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22018.log 2024-09-14T07:13:06.758 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43244.log.gz 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34432.log 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43201.log 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40641.log.gz 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.38978.log.gz 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34432.log.gz 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29175.log 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36945.log 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43201.log.gz 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34212.log 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29175.log.gz 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33098.log 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36945.log.gz 2024-09-14T07:13:06.759 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30787.log 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34212.log.gz 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34318.log 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33098.log.gz 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33488.log 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30787.log.gz 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32498.log 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34318.log.gz 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33333.log 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33488.log.gz 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42832.log 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32498.log.gz 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42993.log 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33333.log.gz 2024-09-14T07:13:06.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28636.log 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42832.log.gz 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31726.log 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42993.log.gz 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28962.log 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28636.log.gz 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42690.log 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31726.log.gz 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35825.log 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28962.log.gz 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35412.log 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gz 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35825.log.gz 2024-09-14T07:13:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42941.log 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35412.log.gz 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37754.log 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42941.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42941.log.gz 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29342.log 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37754.log.gz 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28082.log 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31015.log 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29342.log.gz 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28082.log.gz 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36410.log 2024-09-14T07:13:06.762 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28732.log 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr: 28.1% -- replaced with /var/log/ceph/ceph-client.admin.31015.log.gz 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44267.log 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36410.log.gz 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33254.log 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28732.log.gz 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44267.log.gz 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44241.log 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33382.log 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33254.log.gz 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44241.log.gz 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28584.log 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38188.log 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33382.log.gz 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28584.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.28584.log.gz -5 2024-09-14T07:13:06.763 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36043.log 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44293.log 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38188.log.gz 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36043.log.gz 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28888.log 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37493.log 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44293.log.gz 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28888.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.28888.log.gz 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30708.log 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37493.log.gz 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43071.log 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30708.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30708.log.gz 2024-09-14T07:13:06.764 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42754.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28127.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43071.log.gz 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37410.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42754.log.gz 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33250.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28127.log.gz 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40808.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37410.log.gz 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29217.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33250.log.gz 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37335.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40808.log.gz 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35036.log 2024-09-14T07:13:06.765 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29217.log.gz 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37335.log.gz 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43695.log 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44163.log 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35036.log.gz 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32104.log 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43695.log.gz 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31854.log 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44163.log.gz 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30353.log 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32104.log.gz 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38126.log 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31854.log.gz 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33020.log 2024-09-14T07:13:06.766 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30353.log.gz 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33196.log 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38126.log.gz 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29676.log 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33020.log.gz 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36247.log 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33196.log.gz 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29477.log 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29676.log.gz 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36247.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36247.log.gz -5 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42915.log 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29477.log.gz 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31431.log 2024-09-14T07:13:06.767 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28635.log 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42915.log.gz 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31431.log.gz 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28432.log 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34947.log 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28635.log.gz 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.28432.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32673.log 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28432.log.gz 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43019.log 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34947.log.gz 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34255.log 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32673.log.gz 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44189.log 2024-09-14T07:13:06.768 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43019.log.gz 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31079.log 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34255.log.gz 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38316.log 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44189.log.gz 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31580.log 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31079.log.gz 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43929.log 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38316.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38316.log.gz 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28304.log 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31580.log.gz 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35062.log 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43929.log.gz 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38066.log 2024-09-14T07:13:06.769 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28304.log.gz 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36820.log 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35062.log.gz 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43149.log 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38066.log.gz 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36820.log: /var/log/ceph/ceph-client.admin.29593.log 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36820.log.gz 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42664.log 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31522.log 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29593.log.gz 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28958.log 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42664.log.gz 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38218.log 2024-09-14T07:13:06.770 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31522.log.gz 2024-09-14T07:13:06.771 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35324.log 2024-09-14T07:13:06.771 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28958.log.gz 2024-09-14T07:13:06.771 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37038.log 2024-09-14T07:13:06.771 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38218.log.gz 2024-09-14T07:13:06.812 INFO:teuthology.orchestra.run.smithi110.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23797.log 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23167.log 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.22018.log.gz 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22299.log 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23797.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23797.log.gz 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22009.log 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23167.log.gz 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22299.log: 56.5% -- replaced with /var/log/ceph/ceph-client.admin.22299.log.gz 2024-09-14T07:13:06.888 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23551.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24156.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22009.log.gz 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22966.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23551.log.gz 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24532.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.24156.log.gz 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23210.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24532.log: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.22966.log.gz 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23341.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24532.log.gz 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21939.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.23210.log.gz 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24358.log 2024-09-14T07:13:06.889 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23341.log.gz 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24611.log 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21939.log.gz 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23587.log 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24358.log.gz 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21984.log 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.24611.log.gz 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22887.log 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23587.log.gz 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24016.log 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr: 52.4% -- replaced with /var/log/ceph/ceph-client.admin.21984.log.gz 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24437.log 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22887.log.gz 2024-09-14T07:13:06.890 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22466.log 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.admin.24016.log.gz 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23237.log 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.24437.log.gz 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24462.log 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22466.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22466.log.gz 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24120.log 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23237.log.gz 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21844.log 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24462.log.gz 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22676.log 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24120.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.24120.log.gz 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23176.log 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.21844.log.gz 2024-09-14T07:13:06.891 INFO:teuthology.orchestra.run.smithi145.stderr:gzip/var/log/ceph/ceph-client.admin.22676.log: -5 --verbose -- /var/log/ceph/ceph-client.0.21506.log 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22676.log.gz 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24401.log 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.0.21506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24296.log 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.admin.23176.log.gz 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.21506.log.gz 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24401.log: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.24401.log.gz 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22149.log 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24392.log 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24296.log: /var/log/ceph/ceph-client.admin.22149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21729.log 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22149.log.gz 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.24296.log.gz 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21738.log 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24392.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.24392.log.gz 2024-09-14T07:13:06.892 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23377.log 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21729.log.gz 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24251.log 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.21738.log.gz 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23946.log 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23377.log.gz 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23831.log 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24251.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24251.log.gz 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23000.log 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23831.log: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.23946.log.gz 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23831.log.gzgzip 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.23596.log 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23000.log: gzip 78.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.23000.log.gz -- 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr: /var/log/ceph/ceph-client.admin.22369.log 2024-09-14T07:13:06.893 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22789.log 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23596.log: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.23596.log.gz 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22746.log 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.22369.log.gz 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24568.log 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.22789.log.gz 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23901.log 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22746.log.gz 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24111.log 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24568.log.gz 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23901.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23901.log.gz 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23456.log 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23411.log 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24111.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.24111.log.gz 2024-09-14T07:13:06.894 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21695.log 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.23456.log.gz 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22957.log 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23411.log.gz 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22158.log 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21695.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21695.log.gz 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22265.log 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22957.log.gz 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23350.log 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr: 56.2% -- replaced with /var/log/ceph/ceph-client.admin.22158.log.gz 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23447.log 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22500.log 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr: 53.4% -- replaced with /var/log/ceph/ceph-client.admin.23350.log.gz 2024-09-14T07:13:06.895 INFO:teuthology.orchestra.run.smithi145.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.22265.log.gz 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24428.log 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23447.log.gz 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24147.log 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22500.log.gz 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24428.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24428.log.gz 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22326.log 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23621.log 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24147.log.gz 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22194.log 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22326.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22326.log.gz 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24367.log 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23621.log.gz 2024-09-14T07:13:06.896 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23727.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.admin.22194.log.gz 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22921.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24367.log: /var/log/ceph/ceph-client.admin.23727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21536.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23727.log.gz 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.24367.log.gz 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24217.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22921.log.gz 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22896.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.21536.log.gz 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24217.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24217.log.gz 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22685.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22228.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22896.log: /var/log/ceph/ceph-client.admin.22685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22045.log 2024-09-14T07:13:06.897 INFO:teuthology.orchestra.run.smithi145.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.admin.22896.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.22685.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22439.log 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22228.log: /var/log/ceph/ceph-client.admin.22045.log: 55.5% -- replaced with /var/log/ceph/ceph-client.admin.22228.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24638.log 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22045.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24498.log 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.22439.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23840.log 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24638.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24577.log 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24498.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21589.log 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr: 52.6% -- replaced with /var/log/ceph/ceph-client.admin.23840.log.gz 2024-09-14T07:13:06.898 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23246.log 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.24577.log.gz 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23867.log 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21589.log.gz 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22219.log 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.23246.log.gz 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24541.log 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23867.log.gz 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23770.log 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22219.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22219.log.gz 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24507.log 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr: 54.8% -- replaced with /var/log/ceph/ceph-client.admin.24541.log.gz 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23307.log 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.23770.log.gz 2024-09-14T07:13:06.899 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23027.log 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24507.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.24507.log.gz 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22606.log 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23307.log.gz 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22719.log 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23027.log.gz 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23490.log 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22606.log.gz 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23691.log 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.22719.log.gz 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22991.log 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.23490.log.gz 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22430.log 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23691.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23691.log.gz 2024-09-14T07:13:06.900 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22755.log 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22991.log.gz 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23061.log 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22430.log.gz 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23630.log 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22755.log: /var/log/ceph/ceph-client.admin.23061.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.22755.log.gz 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21704.log 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23061.log.gz 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23271.log 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr: 26.5% -- replaced with /var/log/ceph/ceph-client.admin.23630.log.gz 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23657.log 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21704.log: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.21704.log.gz 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22290.log 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23271.log.gz 2024-09-14T07:13:06.901 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23657.log: gzip 19.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.23657.log.gz -- 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr: /var/log/ceph/ceph-client.admin.24321.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22054.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22290.log.gz 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24321.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24321.log.gz 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23666.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22475.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr: 54.1% -- replaced with /var/log/ceph/ceph-client.admin.22054.log.gz 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21598.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23666.log: /var/log/ceph/ceph-client.admin.22475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23806.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.23666.log.gz 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.22475.log.gz 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23526.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21598.log: /var/log/ceph/ceph-client.admin.23806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22615.log 2024-09-14T07:13:06.902 INFO:teuthology.orchestra.run.smithi145.stderr: 95.5% -- replaced with /var/log/ceph/ceph-client.admin.21598.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.23806.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22817.log 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.23526.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22826.log 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.22615.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22649.log 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22817.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23201.log 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22649.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.22826.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21905.log 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.22649.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24330.log 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23201.log.gz 2024-09-14T07:13:06.903 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23761.log 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21905.log.gz 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24041.log 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr: 35.1% -- replaced with /var/log/ceph/ceph-client.admin.24330.log.gz 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23420.log 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23761.log.gz 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24647.log 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24041.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24041.log.gz 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22579.log 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.23420.log.gz 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21774.log 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr: 72.3% -- replaced with /var/log/ceph/ceph-client.admin.24647.log.gz 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24007.log 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22579.log: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.22579.log.gz 2024-09-14T07:13:06.904 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21975.log 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.21774.log.gz 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22088.log 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24007.log.gz 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24602.log 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21975.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21975.log.gz 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22088.log: gzip 54.0% -- replaced with /var/log/ceph/ceph-client.admin.22088.log.gz -5 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr: --verbose -- /var/log/ceph/ceph-client.admin.22256.log 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21667.log 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24602.log.gz 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22115.log 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22256.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22256.log.gz 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23876.log 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr: 56.0% -- replaced with /var/log/ceph/ceph-client.admin.21667.log.gz 2024-09-14T07:13:06.905 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21799.log 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22115.log.gz 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22545.log 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23876.log: 56.4% -- replaced with /var/log/ceph/ceph-client.admin.23876.log.gz 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21799.log: gzip 19.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.21799.log.gz --verbose 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr: -- /var/log/ceph/ceph-client.admin.23386.log 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22360.log 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr: 94.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.22545.log.gz 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.22640.log 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23386.log: /var/log/ceph/ceph-client.admin.22360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23140.log 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22360.log.gz 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.23386.log.gz 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22079.log 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22640.log.gz 2024-09-14T07:13:06.906 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23280.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.23140.log.gz 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21878.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22079.log.gz 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21633.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr: 51.7% -- replaced with /var/log/ceph/ceph-client.admin.23280.log.gz 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22124.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.21878.log.gz 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23736.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21633.log: 52.1% -- replaced with /var/log/ceph/ceph-client.admin.21633.log.gz 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23517.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23736.log: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.22124.log.gz 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23131.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.23736.log.gz 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22570.log 2024-09-14T07:13:06.907 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23517.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23517.log.gz 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22536.log 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.23131.log.gz 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22396.log 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22570.log.gz 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.22536.log: /var/log/ceph/ceph-client.admin.24181.log 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22536.log.gz 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22185.log 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22396.log.gz 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21835.log 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.24181.log.gz 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24471.log 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22185.log.gz 2024-09-14T07:13:06.908 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22860.log 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21835.log.gz 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24287.log 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.24471.log.gz 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23937.log 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22860.log: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.22860.log.gz 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21914.log 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr: 18.4% -- replaced with /var/log/ceph/ceph-client.admin.24287.log.gz 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23980.log 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23937.log.gz 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23910.log 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21914.log: /var/log/ceph/ceph-client.admin.23980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24190.log 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.21914.log.gz 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr: 57.3% -- replaced with /var/log/ceph/ceph-client.admin.23980.log.gz 2024-09-14T07:13:06.909 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24086.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23910.log.gz 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22851.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.24190.log.gz 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24260.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.24086.log.gz 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21658.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22851.log.gz 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21869.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24260.log: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.24260.log.gz 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24226.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr: 18.4% -- replaced with /var/log/ceph/ceph-client.admin.21658.log.gz 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22509.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21869.log.gz 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22335.log 2024-09-14T07:13:06.910 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24226.log: /var/log/ceph/ceph-client.admin.22509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23070.log 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.24226.log.gz 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.22509.log.gz 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23481.log 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.22335.log.gz 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23560.log 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23070.log: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.23070.log.gz 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21624.log 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23481.log.gz 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24050.log 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.23560.log.gz 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21948.log 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21624.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21624.log.gz 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23316.log 2024-09-14T07:13:06.911 INFO:teuthology.orchestra.run.smithi145.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.24050.log.gz 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22930.log 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr: 56.4% -- replaced with /var/log/ceph/ceph-client.admin.21948.log.gz 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22405.log 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23316.log: 60.6% -- replaced with /var/log/ceph/ceph-client.admin.23316.log.gz 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23036.log 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.22930.log.gz 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21808.log 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 96.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.22405.log.gz -- 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr: /var/log/ceph/ceph-client.admin.23097.log 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23036.log: /var/log/ceph/ceph-client.admin.21808.log: 52.0% -- replaced with /var/log/ceph/ceph-client.admin.21808.log.gz 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23971.log 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.23036.log.gz 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24077.log 2024-09-14T07:13:06.912 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23097.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23971.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.23971.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21527.log 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23106.log 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.24077.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.24077.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.21527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21765.log 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21527.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23106.log: gzip -5 --verbose -- 57.3% /var/log/ceph/ceph-client.admin.23700.log -- replaced with /var/log/ceph/ceph-client.admin.23106.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:gzip/var/log/ceph/ceph-client.admin.21765.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.22780.log 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.21765.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.23700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22710.log 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr: 55.1% -- replaced with /var/log/ceph/ceph-client.admin.23700.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22780.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.22780.log.gz 2024-09-14T07:13:06.913 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.22710.log: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.22710.log.gz 2024-09-14T07:13:06.914 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-09-14T07:13:06.914 INFO:teuthology.orchestra.run.smithi145.stderr:real 0m0.083s 2024-09-14T07:13:06.914 INFO:teuthology.orchestra.run.smithi145.stderr:user 0m0.146s 2024-09-14T07:13:06.914 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m0.042s 2024-09-14T07:13:06.938 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38378.log 2024-09-14T07:13:06.938 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35324.log.gz 2024-09-14T07:13:06.938 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29338.log 2024-09-14T07:13:06.938 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37038.log.gz 2024-09-14T07:13:06.938 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38378.log: /var/log/ceph/ceph-client.admin.35116.log 2024-09-14T07:13:06.938 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38378.log.gz 2024-09-14T07:13:06.938 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33654.log 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29338.log.gz 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43403.log 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35116.log.gz 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42613.log 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33654.log.gz 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43403.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21440.log 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43403.log.gz 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34568.log 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42613.log.gz 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.tmp-client.admin.21440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32797.log 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21440.log.gz 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28580.log 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34568.log.gz 2024-09-14T07:13:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35365.log 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32797.log.gz 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.28580.log: -- /var/log/ceph/ceph-client.admin.29881.log 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28580.log.gz 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36721.log 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35365.log.gz 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37331.log 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29881.log.gz 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36721.log.gz 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44137.log 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37331.log.gz 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32412.log 2024-09-14T07:13:06.940 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44137.log: /var/log/ceph/ceph-client.admin.36494.log 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44137.log.gz 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32890.log 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32412.log.gz 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36494.log.gz 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28273.log 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37679.log 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32890.log.gz 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44317.log 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28273.log.gz 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42338.log 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37679.log.gz 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37758.log 2024-09-14T07:13:06.941 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44317.log.gz 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43851.log 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42338.log.gz 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34280.log 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37758.log.gz 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29651.log 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43851.log.gz 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44215.log 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34280.log.gz 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29843.log 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29651.log.gz 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34104.log 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44215.log.gz 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33621.log 2024-09-14T07:13:06.942 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29843.log.gz 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38279.log 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34104.log.gz 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.33621.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35685.log 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33621.log.gz 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35490.log 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38279.log.gz 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42165.log 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35685.log.gz 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35490.log.gz 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31109.log 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42165.log.gz 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28608.log 2024-09-14T07:13:06.943 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32835.log 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31109.log.gz 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33160.log 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28608.log.gz 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28431.log 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32835.log.gz 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32159.log 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33160.log.gz 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25038.log 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28431.log.gz 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31118.log 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32159.log.gz 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.25038.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-09-14T07:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25038.log.gz 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38498.log 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31118.log.gz 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32601.log 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36190.log 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38498.log.gz 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32601.log.gz 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30328.log 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36190.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36190.log.gz 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30328.log.gz 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36684.log 2024-09-14T07:13:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38976.log 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44033.log 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36684.log.gz 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38976.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38413.log 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38976.log.gz 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28120.log 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44033.log.gz 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28427.log 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38413.log.gz 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31953.log 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28120.log.gz 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33484.log 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28427.log.gz 2024-09-14T07:13:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33436.log 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31953.log.gz 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34686.log 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33484.log.gz 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33436.log.gz 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32974.log 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34686.log.gz 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33539.log 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43543.log 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32974.log.gz 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35286.log 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33539.log.gz 2024-09-14T07:13:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37231.log 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43543.log.gz 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34618.log 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35286.log.gz 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37231.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37231.log.gz 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32374.log 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34618.log.gz 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30517.log 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28758.log 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32374.log.gz 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29213.log 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30517.log.gz 2024-09-14T07:13:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35621.log 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28758.log.gz 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29964.log/var/log/ceph/ceph-client.admin.29213.log: 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29213.log.gz 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35621.log.gz 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29077.log 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29964.log.gz 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28420.log 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29077.log.gz 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30754.log 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.28420.log: -- /var/log/ceph/ceph-client.admin.31386.log 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28420.log.gz 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42447.log 2024-09-14T07:13:06.949 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30754.log.gz 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31912.log 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31386.log.gz 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33791.log 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42447.log.gz 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25232.log 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31912.log.gz 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43391.log 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33791.log.gz 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.25232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34078.log 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25232.log.gz 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43391.log.gz 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39818.log 2024-09-14T07:13:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34078.log.gz 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28113.log 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5/var/log/ceph/ceph-client.admin.39818.log: --verbose -- /var/log/ceph/ceph-client.admin.30655.log 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39818.log.gz 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38092.log 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28113.log.gz 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43595.log 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30655.log.gz 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42539.log 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38092.log.gz 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35850.log 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43595.log.gz 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42539.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T07:13:06.951 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42539.log.gz 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43396.log 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35850.log.gz 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5/var/log/ceph/ceph.log: --verbose -- /var/log/ceph/ceph-client.admin.30395.log 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36134.log 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43396.log.gz 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40682.log 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30395.log.gz 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28862.log 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36134.log: 0.0%/var/log/ceph/ceph-client.admin.40682.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.36134.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.42388.log 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40682.log.gz 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28862.log.gz 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34499.log 2024-09-14T07:13:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44007.log 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42388.log.gz 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34499.log.gz 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37970.log 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43903.log 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44007.log.gz 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32336.log 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37970.log.gz 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34943.log 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43903.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43903.log.gz 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32336.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.44111.log 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32336.log.gz 2024-09-14T07:13:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37040.log 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34943.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34943.log.gz 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31222.log 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44111.log.gz 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34834.log 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37040.log.gz 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31222.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.31222.log.gz /var/log/ceph/ceph-client.admin.37107.log 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32129.log 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34834.log.gz 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28224.log 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37107.log.gz 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32129.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32129.log.gz -5 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-09-14T07:13:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31291.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28224.log.gz 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32725.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42585.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31291.log.gz 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32725.log.gz 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42585.log.gz 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43243.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34108.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gz 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37530.log 2024-09-14T07:13:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34108.log.gz 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27934.log 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37530.log.gz 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30879.log 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27745.log/var/log/ceph/ceph-client.admin.27934.log: 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27934.log.gz 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30879.log.gz 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32473.log 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.27745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.27745.log.gz 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33122.log 2024-09-14T07:13:06.956 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29525.log 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32473.log.gz 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31192.log 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33122.log.gz 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29525.log.gz 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31192.log.gz 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28460.log 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31556.log 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28836.log 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28460.log.gz 2024-09-14T07:13:06.957 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41913.log 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31556.log.gz 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28836.log.gz 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43331.log 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41913.log: /var/log/ceph/ceph-client.admin.43331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28265.log 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41913.log.gz 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43331.log.gz 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42008.log 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30229.log 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28265.log: gzip/var/log/ceph/ceph-client.admin.42008.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42132.log 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42008.log.gz 2024-09-14T07:13:06.958 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28265.log.gz 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30229.log.gz 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36527.log 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37281.log 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42132.log: /var/log/ceph/ceph-client.admin.36527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42132.log.gz 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36527.log.gz 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42055.log 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37281.log.gz 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42889.log 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43773.log: /var/log/ceph/ceph-client.admin.42055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36982.log 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42055.log.gz 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32937.log 2024-09-14T07:13:06.959 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42889.log.gz 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40977.log 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36982.log: /var/log/ceph/ceph-client.admin.32937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31672.log 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36982.log.gz 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr: 27.8% -- replaced with /var/log/ceph/ceph-client.admin.32937.log.gz/var/log/ceph/ceph-client.admin.40977.log: 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40977.log.gz 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.28576.log 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30795.log 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31672.log: /var/log/ceph/ceph-client.admin.28576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28030.log 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31672.log.gz 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28576.log.gz 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36013.log 0.0% 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30795.log.gz 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35153.log 2024-09-14T07:13:06.960 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28030.log: /var/log/ceph/ceph-client.admin.36013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29990.log 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28030.log.gz 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36013.log.gz 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35153.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35153.log.gz 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36130.log 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37141.log 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29990.log: /var/log/ceph/ceph-client.admin.36130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32544.log 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29990.log.gz 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36130.log.gz 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35461.log 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37141.log.gz 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34369.log 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32544.log: /var/log/ceph/ceph-client.admin.35461.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32544.log.gz 2024-09-14T07:13:06.961 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35461.log.gz 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34369.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.34369.log.gz /var/log/ceph/ceph-client.admin.32007.log 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.32007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37903.log 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36871.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32760.log 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38535.log 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37903.log: /var/log/ceph/ceph-client.admin.32760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35586.log 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37903.log.gz 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32760.log.gz 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29754.log 2024-09-14T07:13:06.962 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38535.log.gz 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35586.log.gz 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30904.log 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43877.log 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29754.log: /var/log/ceph/ceph-client.admin.30904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35768.log 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29754.log.gz 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30904.log.gz 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43877.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43877.log.gz 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33890.log 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42720.log 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35768.log: /var/log/ceph/ceph-client.admin.33890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29555.log 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35768.log.gz 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33890.log.gz 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42720.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.42720.log.gz --verbose 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.36282.log 2024-09-14T07:13:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29555.log: /var/log/ceph/ceph-client.admin.36282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43539.log 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29555.log.gz 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36282.log.gz 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43747.log 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42094.log 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43539.log: /var/log/ceph/ceph-client.admin.43747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43539.log.gz 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43747.log.gz 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42094.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42094.log.gz /var/log/ceph/ceph-client.admin.42392.log 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43175.log 2024-09-14T07:13:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34724.log: /var/log/ceph/ceph-client.admin.42392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43238.log 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43175.log.gz 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28056.log 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30493.log 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43238.log: /var/log/ceph/ceph-client.admin.28056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28784.log 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28056.log.gz 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43238.log.gz 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30493.log.gz 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36317.log 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39854.log 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28784.log: /var/log/ceph/ceph-client.admin.36317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33411.log 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36317.log.gz 2024-09-14T07:13:06.965 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28784.log.gz 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39854.log.gz 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32057.log 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30618.log 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33411.log: /var/log/ceph/ceph-client.admin.32057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28128.log 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32057.log.gz 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33411.log.gz 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30618.log.gz 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43045.log 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29813.log 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28128.log: /var/log/ceph/ceph-client.admin.43045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34592.log 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43045.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28128.log.gz 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29813.log.gz 2024-09-14T07:13:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32134.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34592.log: /var/log/ceph/ceph-client.admin.32134.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.38527.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34592.log.gz 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32134.log.gz 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31957.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43097.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/objectstore_tool.38527.log: /var/log/ceph/ceph-client.admin.31957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29114.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31957.log.gz 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43097.log.gz 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.38527.log.gz 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30571.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33967.log 2024-09-14T07:13:06.967 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29114.log.gz 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30571.log.gz 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31817.log 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33967.log: /var/log/ceph/ceph-client.admin.43955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33716.log 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33967.log.gz 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31817.log.gz 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42007.log 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35241.log 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33716.log.gz 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31459.log 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42007.log.gz 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43599.log 2024-09-14T07:13:06.968 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35241.log: /var/log/ceph/ceph-client.admin.31459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31316.log 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35241.log.gz 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31459.log.gz 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37592.log 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43599.log.gz 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41938.log 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31316.log: /var/log/ceph/ceph-client.admin.37592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31316.log.gz 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37592.log.gz 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41938.log.gzgzip 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28957.log 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40775.log: /var/log/ceph/ceph-client.admin.43799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40728.log 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-09-14T07:13:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28957.log.gzgzip 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36445.log 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33073.log 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40728.log: /var/log/ceph/ceph-client.admin.36445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29437.log 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40728.log.gz 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36445.log.gz 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33073.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33073.log.gz 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43123.log 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36846.log 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.29437.log: /var/log/ceph/ceph-client.admin.43123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35968.log 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43123.log.gz 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29437.log.gz 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36846.log.gz 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36215.log 2024-09-14T07:13:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34774.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35968.log: /var/log/ceph/ceph-client.admin.36215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28810.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35968.log.gz 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36215.log.gz 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31043.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34774.log.gz 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30150.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28810.log: /var/log/ceph/ceph-client.admin.31043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42296.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28810.log.gz 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31043.log.gz 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30150.log.gz 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27959.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30978.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42296.log: /var/log/ceph/ceph-client.admin.27959.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.38778.log 2024-09-14T07:13:06.971 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42296.log.gz 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27959.log.gz 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.30978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30978.log.gz 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33720.log 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31639.log 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/objectstore_tool.38778.log: /var/log/ceph/ceph-client.admin.33720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32273.log 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33720.log.gz 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31639.log.gz 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr: 94.8% -- replaced with /var/log/ceph/objectstore_tool.38778.log.gz 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34469.log 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33840.log 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32273.log.gz 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28276.log 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34469.log: /var/log/ceph/ceph-client.admin.33840.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34012.log 2024-09-14T07:13:06.972 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33840.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34469.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.28276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31768.log 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28276.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32833.log 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34012.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29750.log 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31768.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43548.log 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32833.log: /var/log/ceph/ceph-client.admin.29750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34322.log 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32833.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29750.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28303.log 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43548.log.gz 2024-09-14T07:13:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44059.log 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34322.log: /var/log/ceph/ceph-client.admin.28303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43247.log 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34322.log.gz 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28303.log.gz 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42215.log 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35002.log 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43247.log: /var/log/ceph/ceph-client.admin.42215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43247.log.gz 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42215.log.gz 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35002.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.35002.log.gz --verbose 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: -- /var/log/ceph/ceph-client.admin.37806.log 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36348.log 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41415.log: /var/log/ceph/ceph-client.admin.37806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37806.log.gz 2024-09-14T07:13:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36348.log.gz 2024-09-14T07:13:06.975 INFO:teuthology.orchestra.run.smithi110.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T07:13:07.006 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T07:13:07.123 INFO:teuthology.orchestra.run.smithi005.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T07:13:08.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-14T07:13:09.937 INFO:teuthology.orchestra.run.smithi005.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T07:13:10.107 INFO:teuthology.orchestra.run.smithi005.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T07:13:11.047 INFO:teuthology.orchestra.run.smithi005.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T07:13:11.047 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-14T07:13:11.048 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m4.363s 2024-09-14T07:13:11.048 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m11.385s 2024-09-14T07:13:11.048 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.499s 2024-09-14T07:13:13.014 INFO:teuthology.orchestra.run.smithi110.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T07:13:14.159 INFO:teuthology.orchestra.run.smithi110.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T07:13:16.218 INFO:teuthology.orchestra.run.smithi110.stderr: 96.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T07:13:16.219 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-09-14T07:13:16.219 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m9.500s 2024-09-14T07:13:16.219 INFO:teuthology.orchestra.run.smithi110.stderr:user 0m22.485s 2024-09-14T07:13:16.219 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m1.015s 2024-09-14T07:13:16.220 INFO:tasks.ceph:Archiving logs... 2024-09-14T07:13:16.220 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi005/log 2024-09-14T07:13:16.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T07:13:17.653 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi110/log 2024-09-14T07:13:17.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T07:13:19.950 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi145/log 2024-09-14T07:13:19.951 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T07:13:20.094 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T07:13:20.115 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T07:13:20.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T07:13:20.117 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T07:13:20.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T07:13:20.221 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T07:13:20.230 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T07:13:20.231 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:13:20.234 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:13:20.236 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:13:20.244 INFO:teuthology.orchestra.run.smithi005.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T07:13:20.245 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================== 2024-09-14T07:13:20.245 INFO:teuthology.orchestra.run.smithi005.stdout:+hv01.front.sepi 67.205.162.81 3 u 49 64 377 0.109 -0.348 0.719 2024-09-14T07:13:20.245 INFO:teuthology.orchestra.run.smithi005.stdout:+hv02.front.sepi 63.231.80.2 3 u 45 64 377 0.142 -1.762 0.770 2024-09-14T07:13:20.245 INFO:teuthology.orchestra.run.smithi005.stdout:*hv03.front.sepi 74.6.168.72 3 u 49 64 377 0.138 -1.065 0.727 2024-09-14T07:13:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:13:20.247 INFO:teuthology.orchestra.run.smithi145.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T07:13:20.247 INFO:teuthology.orchestra.run.smithi145.stdout:============================================================================== 2024-09-14T07:13:20.247 INFO:teuthology.orchestra.run.smithi145.stdout:*hv01.front.sepi 67.205.162.81 3 u 48 64 377 0.136 -0.271 0.740 2024-09-14T07:13:20.247 INFO:teuthology.orchestra.run.smithi145.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 377 0.082 -1.738 0.717 2024-09-14T07:13:20.247 INFO:teuthology.orchestra.run.smithi145.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.110 -0.987 0.661 2024-09-14T07:13:20.247 INFO:teuthology.orchestra.run.smithi145.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:13:20.248 INFO:teuthology.orchestra.run.smithi110.stdout: remote refid st t when poll reach delay offset jitter 2024-09-14T07:13:20.249 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================== 2024-09-14T07:13:20.249 INFO:teuthology.orchestra.run.smithi110.stdout:*hv01.front.sepi 67.205.162.81 3 u 40 64 377 0.078 +0.288 0.429 2024-09-14T07:13:20.249 INFO:teuthology.orchestra.run.smithi110.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 377 0.100 -1.205 0.533 2024-09-14T07:13:20.249 INFO:teuthology.orchestra.run.smithi110.stdout:+hv03.front.sepi 74.6.168.72 3 u 43 64 377 0.114 -0.484 0.448 2024-09-14T07:13:20.249 INFO:teuthology.orchestra.run.smithi110.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-14T07:13:20.249 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T07:13:20.259 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T07:13:20.260 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T07:13:20.345 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T07:13:20.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T07:13:20.362 INFO:teuthology.task.internal:Duration was 974.850505 seconds 2024-09-14T07:13:20.363 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T07:13:20.370 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T07:13:20.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T07:13:20.373 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T07:13:20.375 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T07:13:20.413 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T07:13:20.413 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-09-14T07:13:20.414 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T07:13:20.465 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-14T07:13:20.465 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T07:13:20.475 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-09-14T07:13:20.475 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T07:13:20.485 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T07:13:20.485 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T07:13:20.508 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T07:13:20.520 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T07:13:20.554 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T07:13:20.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T07:13:20.584 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T07:13:20.597 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T07:13:20.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T07:13:20.677 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T07:13:20.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T07:13:20.690 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T07:13:20.700 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T07:13:20.712 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T07:13:20.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:13:20.732 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:13:20.739 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-09-14T07:13:20.744 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:13:20.751 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-14T07:13:20.764 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-09-14T07:13:20.781 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:13:20.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:13:20.811 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:13:20.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:13:20.819 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:13:20.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:13:20.826 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T07:13:20.835 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T07:13:20.836 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi005 2024-09-14T07:13:20.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T07:13:20.877 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi110 2024-09-14T07:13:20.877 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T07:13:20.898 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900414/remote/smithi145 2024-09-14T07:13:20.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T07:13:20.921 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T07:13:20.921 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T07:13:20.924 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T07:13:20.936 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T07:13:20.967 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T07:13:20.978 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T07:13:20.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T07:13:20.988 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T07:13:20.988 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T07:13:20.990 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T07:13:20.992 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T07:13:20.995 INFO:teuthology.orchestra.run.smithi005.stdout: 658464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 07:13 /home/ubuntu/cephtest 2024-09-14T07:13:20.996 INFO:teuthology.orchestra.run.smithi110.stdout: 658642 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 07:13 /home/ubuntu/cephtest 2024-09-14T07:13:21.015 INFO:teuthology.orchestra.run.smithi145.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 07:13 /home/ubuntu/cephtest 2024-09-14T07:13:21.016 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T07:13:21.026 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T07:13:21.052 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 974.8505048751831 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T07:13:21.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T07:13:21.144 INFO:teuthology.run:pass